[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:36:59 dist-test-slave-w4v5 krb5kdc[1029](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:36:59 dist-test-slave-w4v5 krb5kdc[1029](info): set up 2 sockets
Aug 12 16:36:59 dist-test-slave-w4v5 krb5kdc[1029](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260812 16:37:00.636413 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.306s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:37:00 dist-test-slave-w4v5 krb5kdc[1029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:37:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:37:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
I20260812 16:37:00.764163 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:44799
--webserver_interface=127.0.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:39363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:44799
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:00.872627 1062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:00.872857 1062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:00.872881 1062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:00.873976 1062 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:37:00.874014 1062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:00.874028 1062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:00.874042 1062 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:37:00.874054 1062 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:37:00.875663 1062 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:39363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:44799
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:44799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:00.875993 1062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:00.876209 1062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:00.884545 1078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:00.884541 1085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:00.885210 1080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:00.885172 1062 server_base.cc:1034] running on GCE node
I20260812 16:37:00.885581 1062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:00.886005 1062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:00.887154 1062 hybrid_clock.cc:648] HybridClock initialized: now 1786552620887142 us; error 50 us; skew 500 ppm
Aug 12 16:37:00 dist-test-slave-w4v5 krb5kdc[1029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552620, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:00.889925 1062 init.cc:377] Logged in from keytab as kudu/127.0.253.254@KRBTEST.COM (short username kudu)
I20260812 16:37:00.890686 1062 webserver.cc:460] Webserver started at http://127.0.253.254:46183/ using document root <none> and password file <none>
I20260812 16:37:00.890998 1062 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:00.891077 1062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:00.891364 1062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:00.892704 1062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "de3c5793442141efa615500e1f2541c0"
format_stamp: "Formatted at 2026-08-12 16:37:00 on dist-test-slave-w4v5"
server_key: "bad7a2befa31feb78b88a64192025c0b"
server_key_iv: "2d6e2f18707eab35aeb19781de5af179"
server_key_version: "encryptionkey@0"
I20260812 16:37:00.893164 1062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "de3c5793442141efa615500e1f2541c0"
format_stamp: "Formatted at 2026-08-12 16:37:00 on dist-test-slave-w4v5"
server_key: "bad7a2befa31feb78b88a64192025c0b"
server_key_iv: "2d6e2f18707eab35aeb19781de5af179"
server_key_version: "encryptionkey@0"
I20260812 16:37:00.895193 1062 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260812 16:37:00.896312 1104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:00.896631 1062 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:00.896749 1062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "de3c5793442141efa615500e1f2541c0"
format_stamp: "Formatted at 2026-08-12 16:37:00 on dist-test-slave-w4v5"
server_key: "bad7a2befa31feb78b88a64192025c0b"
server_key_iv: "2d6e2f18707eab35aeb19781de5af179"
server_key_version: "encryptionkey@0"
I20260812 16:37:00.896829 1062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:00.925683 1062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:00.928468 1062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:00.928709 1062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:00.934418 1062 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:44799
I20260812 16:37:00.934600 1216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:44799 every 8 connection(s)
I20260812 16:37:00.935223 1062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:37:00.936627 1217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:00.940013 1217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de3c5793442141efa615500e1f2541c0: Bootstrap starting.
I20260812 16:37:00.940951 1217 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de3c5793442141efa615500e1f2541c0: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:00.941256 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 1062
I20260812 16:37:00.941323 1217 log.cc:824] T 00000000000000000000000000000000 P de3c5793442141efa615500e1f2541c0: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:00.941396 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:37:00.942246 1217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de3c5793442141efa615500e1f2541c0: No bootstrap required, opened a new log
I20260812 16:37:00.942253 1015 external_mini_cluster.cc:1428] Setting key 90fd8894d01bd49da1a28c6bb8287621
I20260812 16:37:00.944583 1217 raft_consensus.cc:348] T 00000000000000000000000000000000 P de3c5793442141efa615500e1f2541c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de3c5793442141efa615500e1f2541c0" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 44799 } }
I20260812 16:37:00.944780 1217 raft_consensus.cc:374] T 00000000000000000000000000000000 P de3c5793442141efa615500e1f2541c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:00.944926 1217 raft_consensus.cc:729] T 00000000000000000000000000000000 P de3c5793442141efa615500e1f2541c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de3c5793442141efa615500e1f2541c0, State: Initialized, Role: FOLLOWER
I20260812 16:37:00.945139 1217 consensus_queue.cc:260] T 00000000000000000000000000000000 P de3c5793442141efa615500e1f2541c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de3c5793442141efa615500e1f2541c0" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 44799 } }
I20260812 16:37:00.945343 1217 raft_consensus.cc:388] T 00000000000000000000000000000000 P de3c5793442141efa615500e1f2541c0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:37:00.945411 1217 raft_consensus.cc:482] T 00000000000000000000000000000000 P de3c5793442141efa615500e1f2541c0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:37:00.945462 1217 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de3c5793442141efa615500e1f2541c0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:00.946542 1217 raft_consensus.cc:504] T 00000000000000000000000000000000 P de3c5793442141efa615500e1f2541c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de3c5793442141efa615500e1f2541c0" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 44799 } }
I20260812 16:37:00.946770 1217 leader_election.cc:302] T 00000000000000000000000000000000 P de3c5793442141efa615500e1f2541c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de3c5793442141efa615500e1f2541c0; no voters:
I20260812 16:37:00.947002 1217 leader_election.cc:288] T 00000000000000000000000000000000 P de3c5793442141efa615500e1f2541c0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:37:00.947113 1229 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de3c5793442141efa615500e1f2541c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:37:00.947325 1229 raft_consensus.cc:686] T 00000000000000000000000000000000 P de3c5793442141efa615500e1f2541c0 [term 1 LEADER]: Becoming Leader. State: Replica: de3c5793442141efa615500e1f2541c0, State: Running, Role: LEADER
I20260812 16:37:00.947506 1229 consensus_queue.cc:237] T 00000000000000000000000000000000 P de3c5793442141efa615500e1f2541c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de3c5793442141efa615500e1f2541c0" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 44799 } }
I20260812 16:37:00.947566 1217 sys_catalog.cc:564] T 00000000000000000000000000000000 P de3c5793442141efa615500e1f2541c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:37:00.948212 1234 sys_catalog.cc:455] T 00000000000000000000000000000000 P de3c5793442141efa615500e1f2541c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de3c5793442141efa615500e1f2541c0. Latest consensus state: current_term: 1 leader_uuid: "de3c5793442141efa615500e1f2541c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de3c5793442141efa615500e1f2541c0" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 44799 } } }
I20260812 16:37:00.948300 1234 sys_catalog.cc:458] T 00000000000000000000000000000000 P de3c5793442141efa615500e1f2541c0 [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:00.948388 1231 sys_catalog.cc:455] T 00000000000000000000000000000000 P de3c5793442141efa615500e1f2541c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de3c5793442141efa615500e1f2541c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de3c5793442141efa615500e1f2541c0" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 44799 } } }
I20260812 16:37:00.948447 1231 sys_catalog.cc:458] T 00000000000000000000000000000000 P de3c5793442141efa615500e1f2541c0 [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:00.948658 1243 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:37:00.949875 1243 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:37:00.952839 1243 catalog_manager.cc:1347] Generated new cluster ID: 61c4caf167234c8cbe668f5c90f005b6
I20260812 16:37:00.952906 1243 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 16:37:00 dist-test-slave-w4v5 krb5kdc[1029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:00.960911 1228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:00.944927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39478 (local address 127.0.253.254:44799)
0812 16:37:00.945198 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:00.945218 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:00.945263 (+ 45us) server_negotiation.cc:407] Connection header received
0812 16:37:00.946479 (+ 1216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:00.946487 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:00.946606 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:00.946808 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:00.949743 (+ 2935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:00.951596 (+ 1853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:00.953052 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:00.953516 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:00.953674 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:00.957106 (+ 3432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:00.957110 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:00.957110 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:00.957120 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:00.958826 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:00.959348 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:00.959351 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:00.959351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:00.959453 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:00.959722 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:00.959723 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:00.959723 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:00.959955 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:00.960147 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:00.960418 (+ 271us) server_negotiation.cc:299] Negotiation successful
0812 16:37:00.960614 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":87,"threads_started":1}
I20260812 16:37:00.966017 1243 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:37:00.966796 1243 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:37:00.977176 1243 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de3c5793442141efa615500e1f2541c0: Generated new TSK 0
I20260812 16:37:00.977376 1243 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab.
Entry for principal kudu/127.0.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab.
I20260812 16:37:01.012712 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.193:0
--local_ip_for_outbound_sockets=127.0.253.193
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:44799
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:39363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:01.103377 1262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:01.103587 1262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:01.103613 1262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:01.104820 1262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:01.104866 1262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:01.104893 1262 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.193
I20260812 16:37:01.106381 1262 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:39363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:44799
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1262
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.193
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:01.106716 1262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:01.106978 1262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:01.113848 1270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:01.113968 1272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:01.113931 1269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:01.114349 1262 server_base.cc:1034] running on GCE node
I20260812 16:37:01.114617 1262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:01.114925 1262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:01.116132 1262 hybrid_clock.cc:648] HybridClock initialized: now 1786552621116116 us; error 34 us; skew 500 ppm
Aug 12 16:37:01 dist-test-slave-w4v5 krb5kdc[1029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552621, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:01.119534 1262 init.cc:377] Logged in from keytab as kudu/127.0.253.193@KRBTEST.COM (short username kudu)
I20260812 16:37:01.120069 1262 webserver.cc:460] Webserver started at http://127.0.253.193:44697/ using document root <none> and password file <none>
I20260812 16:37:01.120337 1262 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:01.120409 1262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:01.120558 1262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:01.121789 1262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-0/data/instance:
uuid: "0af206eb411644ffb735b4df3bb580d8"
format_stamp: "Formatted at 2026-08-12 16:37:01 on dist-test-slave-w4v5"
server_key: "b69b27c6818a325390acfb4f1bd9b370"
server_key_iv: "73b163a9015c51d7932d5023ebafc4cd"
server_key_version: "encryptionkey@0"
I20260812 16:37:01.122187 1262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance:
uuid: "0af206eb411644ffb735b4df3bb580d8"
format_stamp: "Formatted at 2026-08-12 16:37:01 on dist-test-slave-w4v5"
server_key: "b69b27c6818a325390acfb4f1bd9b370"
server_key_iv: "73b163a9015c51d7932d5023ebafc4cd"
server_key_version: "encryptionkey@0"
I20260812 16:37:01.123739 1262 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:37:01.124625 1290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:01.124840 1262 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:37:01.124928 1262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-0/wal
uuid: "0af206eb411644ffb735b4df3bb580d8"
format_stamp: "Formatted at 2026-08-12 16:37:01 on dist-test-slave-w4v5"
server_key: "b69b27c6818a325390acfb4f1bd9b370"
server_key_iv: "73b163a9015c51d7932d5023ebafc4cd"
server_key_version: "encryptionkey@0"
I20260812 16:37:01.125020 1262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:01.150723 1262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:01.153141 1262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:01.153326 1262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:01.153661 1262 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:37:01.154083 1262 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:01.154189 1262 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:01.154235 1262 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:01.154265 1262 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:01.161197 1262 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.193:33125
I20260812 16:37:01.161279 1537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.193:33125 every 8 connection(s)
I20260812 16:37:01.161613 1262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
Aug 12 16:37:01 dist-test-slave-w4v5 krb5kdc[1029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552621, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:01.169378 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 1262
I20260812 16:37:01.169503 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance
I20260812 16:37:01.169687 1015 external_mini_cluster.cc:1428] Setting key 9cb10decaba01879ba86d16531f3995a
I20260812 16:37:01.172151 1228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:01.162504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:59587 (local address 127.0.253.254:44799)
0812 16:37:01.162595 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:01.162597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:01.162810 (+ 213us) server_negotiation.cc:407] Connection header received
0812 16:37:01.163364 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:01.163365 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:01.163381 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:01.163443 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:01.164640 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:01.165148 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:01.166251 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:01.166443 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:01.166496 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:01.168837 (+ 2341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:01.168838 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:01.168838 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:01.168844 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:01.170673 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:01.171102 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:01.171103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:01.171104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:01.171176 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:01.171452 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:01.171453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:01.171453 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:01.171572 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:01.171642 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:01.171950 (+ 308us) server_negotiation.cc:299] Negotiation successful
0812 16:37:01.172049 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 16:37:01.172348 1541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:01.162446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:44799 (local address 127.0.253.193:59587)
0812 16:37:01.162646 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:01.162685 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:01.163242 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:01.163529 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:01.163533 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:01.164308 (+ 775us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:01.164522 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:01.164527 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:01.165286 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:01.165287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:01.166160 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:01.166163 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:01.166520 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:01.166521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:01.166670 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:01.167300 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:01.167308 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:01.168740 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:01.170761 (+ 2021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:01.170769 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:01.170772 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:01.171002 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:01.171254 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:01.171254 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:01.171255 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:01.171374 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:01.171655 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:01.171661 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:01.171798 (+ 137us) client_negotiation.cc:769] Sending connection context
0812 16:37:01.171928 (+ 130us) client_negotiation.cc:240] Negotiation successful
0812 16:37:01.172093 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":110,"threads_started":1}
I20260812 16:37:01.172912 1538 heartbeater.cc:347] Connected to a master server at 127.0.253.254:44799
I20260812 16:37:01.173036 1538 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:01.173244 1538 heartbeater.cc:511] Master 127.0.253.254:44799 requested a full tablet report, sending...
I20260812 16:37:01.174201 1136 ts_manager.cc:194] Registered new tserver with Master: 0af206eb411644ffb735b4df3bb580d8 (127.0.253.193:33125)
I20260812 16:37:01.174929 1136 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.193@KRBTEST.COM'} at 127.0.253.193:59587
WARNING: no policy specified for kudu/127.0.253.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab.
Entry for principal kudu/127.0.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab.
I20260812 16:37:01.198757 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.194:0
--local_ip_for_outbound_sockets=127.0.253.194
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:44799
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:39363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:01.285825 1546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:01.286077 1546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:01.286119 1546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:01.287398 1546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:01.287462 1546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:01.287508 1546 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.194
I20260812 16:37:01.289011 1546 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:39363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:44799
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.194
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:01.289287 1546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:01.289525 1546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:01.294911 1561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:01.294912 1562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:01.294912 1568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:01.295361 1546 server_base.cc:1034] running on GCE node
I20260812 16:37:01.295567 1546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:01.295814 1546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:01.296988 1546 hybrid_clock.cc:648] HybridClock initialized: now 1786552621296965 us; error 42 us; skew 500 ppm
Aug 12 16:37:01 dist-test-slave-w4v5 krb5kdc[1029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552621, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:01.299741 1546 init.cc:377] Logged in from keytab as kudu/127.0.253.194@KRBTEST.COM (short username kudu)
I20260812 16:37:01.300292 1546 webserver.cc:460] Webserver started at http://127.0.253.194:43089/ using document root <none> and password file <none>
I20260812 16:37:01.300532 1546 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:01.300599 1546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:01.300738 1546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:01.301909 1546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-1/data/instance:
uuid: "f1dbbcb6a1334c58a44ec0cae8fd02aa"
format_stamp: "Formatted at 2026-08-12 16:37:01 on dist-test-slave-w4v5"
server_key: "032754f4df197d7c0c31b1a3d027fa76"
server_key_iv: "369b1df7ba492bdce7b81b919586c2d6"
server_key_version: "encryptionkey@0"
I20260812 16:37:01.302290 1546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance:
uuid: "f1dbbcb6a1334c58a44ec0cae8fd02aa"
format_stamp: "Formatted at 2026-08-12 16:37:01 on dist-test-slave-w4v5"
server_key: "032754f4df197d7c0c31b1a3d027fa76"
server_key_iv: "369b1df7ba492bdce7b81b919586c2d6"
server_key_version: "encryptionkey@0"
I20260812 16:37:01.303715 1546 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:37:01.304457 1588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:01.304636 1546 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:37:01.304745 1546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-1/wal
uuid: "f1dbbcb6a1334c58a44ec0cae8fd02aa"
format_stamp: "Formatted at 2026-08-12 16:37:01 on dist-test-slave-w4v5"
server_key: "032754f4df197d7c0c31b1a3d027fa76"
server_key_iv: "369b1df7ba492bdce7b81b919586c2d6"
server_key_version: "encryptionkey@0"
I20260812 16:37:01.304821 1546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:01.321585 1546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:01.323773 1546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:01.323958 1546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:01.324241 1546 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:37:01.324596 1546 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:01.324677 1546 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:01.324723 1546 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:01.324760 1546 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:01.332172 1546 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.194:41659
I20260812 16:37:01.332216 1836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.194:41659 every 8 connection(s)
I20260812 16:37:01.332631 1546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
I20260812 16:37:01.334873 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 1546
I20260812 16:37:01.334986 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance
I20260812 16:37:01.335191 1015 external_mini_cluster.cc:1428] Setting key 290d7edef5335756261b9b89fa0dd05c
Aug 12 16:37:01 dist-test-slave-w4v5 krb5kdc[1029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552621, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:01.343832 1228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:01.333410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:52061 (local address 127.0.253.254:44799)
0812 16:37:01.333498 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:01.333501 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:01.333772 (+ 271us) server_negotiation.cc:407] Connection header received
0812 16:37:01.334343 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:01.334344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:01.334355 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:01.334404 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:01.335785 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:01.336398 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:01.337595 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:01.337875 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:01.337935 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:01.340650 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:01.340652 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:01.340652 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:01.340661 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:01.342181 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:01.342643 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:01.342644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:01.342645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:01.342715 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:01.343008 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:01.343009 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:01.343009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:01.343192 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:01.343274 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:01.343611 (+ 337us) server_negotiation.cc:299] Negotiation successful
0812 16:37:01.343728 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 16:37:01.344020 1842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:01.333380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:44799 (local address 127.0.253.194:52061)
0812 16:37:01.333602 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:01.333659 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:01.334248 (+ 589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:01.334472 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:01.334476 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:01.335404 (+ 928us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:01.335675 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:01.335681 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:01.336512 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:01.336513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:01.337444 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:01.337446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:01.337971 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:01.337972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:01.338158 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:01.338780 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:01.338794 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:01.340530 (+ 1736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:01.342276 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:01.342285 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:01.342288 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:01.342547 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:01.342803 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:01.342804 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:01.342804 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:01.342933 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:01.343277 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:01.343286 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:01.343447 (+ 161us) client_negotiation.cc:769] Sending connection context
0812 16:37:01.343564 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 16:37:01.343753 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":110,"threads_started":1}
I20260812 16:37:01.344612 1837 heartbeater.cc:347] Connected to a master server at 127.0.253.254:44799
I20260812 16:37:01.344717 1837 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.0.253.195@KRBTEST.COM; defaulting to no policy
I20260812 16:37:01.344965 1837 heartbeater.cc:511] Master 127.0.253.254:44799 requested a full tablet report, sending...
I20260812 16:37:01.345404 1136 ts_manager.cc:194] Registered new tserver with Master: f1dbbcb6a1334c58a44ec0cae8fd02aa (127.0.253.194:41659)
I20260812 16:37:01.345764 1136 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.194@KRBTEST.COM'} at 127.0.253.194:52061
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab.
Entry for principal kudu/127.0.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab.
I20260812 16:37:01.364171 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.195:0
--local_ip_for_outbound_sockets=127.0.253.195
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:44799
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:39363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:01.448760 1847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:01.448971 1847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:01.448995 1847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:01.450258 1847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:01.450323 1847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:01.450369 1847 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.195
I20260812 16:37:01.451948 1847 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:39363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:44799
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.195
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:01.452227 1847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:01.452461 1847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:01.457656 1862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:01.457731 1871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:01.457681 1860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:01.457980 1847 server_base.cc:1034] running on GCE node
I20260812 16:37:01.458196 1847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:01.458455 1847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:01.459617 1847 hybrid_clock.cc:648] HybridClock initialized: now 1786552621459610 us; error 42 us; skew 500 ppm
Aug 12 16:37:01 dist-test-slave-w4v5 krb5kdc[1029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552621, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:01.462468 1847 init.cc:377] Logged in from keytab as kudu/127.0.253.195@KRBTEST.COM (short username kudu)
I20260812 16:37:01.462970 1847 webserver.cc:460] Webserver started at http://127.0.253.195:42241/ using document root <none> and password file <none>
I20260812 16:37:01.463258 1847 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:01.463330 1847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:01.463487 1847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:01.464520 1847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-2/data/instance:
uuid: "8fcaa9fa609a4bd0bcb7faa24b1d47aa"
format_stamp: "Formatted at 2026-08-12 16:37:01 on dist-test-slave-w4v5"
server_key: "5efb0934c080109717044c3897ee5f81"
server_key_iv: "ac55f2778fabc86d601a3ed29722c1e6"
server_key_version: "encryptionkey@0"
I20260812 16:37:01.464901 1847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance:
uuid: "8fcaa9fa609a4bd0bcb7faa24b1d47aa"
format_stamp: "Formatted at 2026-08-12 16:37:01 on dist-test-slave-w4v5"
server_key: "5efb0934c080109717044c3897ee5f81"
server_key_iv: "ac55f2778fabc86d601a3ed29722c1e6"
server_key_version: "encryptionkey@0"
I20260812 16:37:01.466265 1847 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:37:01.467160 1891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:01.467338 1847 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:01.467442 1847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-2/wal
uuid: "8fcaa9fa609a4bd0bcb7faa24b1d47aa"
format_stamp: "Formatted at 2026-08-12 16:37:01 on dist-test-slave-w4v5"
server_key: "5efb0934c080109717044c3897ee5f81"
server_key_iv: "ac55f2778fabc86d601a3ed29722c1e6"
server_key_version: "encryptionkey@0"
I20260812 16:37:01.467522 1847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:01.484439 1847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:01.486809 1847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:01.486970 1847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:01.487233 1847 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:37:01.487555 1847 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:01.487620 1847 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:01.487669 1847 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:01.487697 1847 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:01.493883 1847 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.195:33009
I20260812 16:37:01.493963 2122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.195:33009 every 8 connection(s)
I20260812 16:37:01.494292 1847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
I20260812 16:37:01.499531 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 1847
I20260812 16:37:01.499632 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance
I20260812 16:37:01.499806 1015 external_mini_cluster.cc:1428] Setting key 74d1231eeaaa3abd3d2e6612bdc475ab
Aug 12 16:37:01 dist-test-slave-w4v5 krb5kdc[1029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552621, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:01.503921 1228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:01.495044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:34409 (local address 127.0.253.254:44799)
0812 16:37:01.495109 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:01.495111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:01.495301 (+ 190us) server_negotiation.cc:407] Connection header received
0812 16:37:01.495816 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:01.495817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:01.495828 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:01.495888 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:01.496827 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:01.497328 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:01.498284 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:01.498461 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:01.498513 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:01.501114 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:01.501116 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:01.501117 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:01.501124 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:01.502618 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:01.502978 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:01.502979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:01.502980 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:01.503031 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:01.503309 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:01.503310 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:01.503310 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:01.503416 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:01.503471 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:01.503774 (+ 303us) server_negotiation.cc:299] Negotiation successful
0812 16:37:01.503838 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 16:37:01.504144 2133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:01.495001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:44799 (local address 127.0.253.195:34409)
0812 16:37:01.495187 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:01.495229 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:01.495723 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:01.495966 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:01.495970 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:01.496547 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:01.496740 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:01.496745 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:01.497409 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:01.497410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:01.498208 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:01.498210 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:01.498540 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:01.498541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:01.498644 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:01.499169 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:01.499179 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:01.500988 (+ 1809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:01.502693 (+ 1705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:01.502698 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:01.502700 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:01.502911 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:01.503135 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:01.503136 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:01.503136 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:01.503245 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:01.503505 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:01.503512 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:01.503636 (+ 124us) client_negotiation.cc:769] Sending connection context
0812 16:37:01.503745 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 16:37:01.503873 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":111,"threads_started":1}
I20260812 16:37:01.504635 2123 heartbeater.cc:347] Connected to a master server at 127.0.253.254:44799
I20260812 16:37:01.504801 2123 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:01.505012 2123 heartbeater.cc:511] Master 127.0.253.254:44799 requested a full tablet report, sending...
I20260812 16:37:01.505383 1136 ts_manager.cc:194] Registered new tserver with Master: 8fcaa9fa609a4bd0bcb7faa24b1d47aa (127.0.253.195:33009)
I20260812 16:37:01.505777 1136 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.195@KRBTEST.COM'} at 127.0.253.195:34409
I20260812 16:37:01.512373 1015 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 16:37:01 dist-test-slave-w4v5 krb5kdc[1029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552621, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 16:37:01 dist-test-slave-w4v5 krb5kdc[1029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552621, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.253.193@KRBTEST.COM
I20260812 16:37:01.538353 2144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:01.530188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59630 (local address 127.0.253.193:33125)
0812 16:37:01.530370 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:01.530374 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:01.530387 (+ 13us) server_negotiation.cc:407] Connection header received
0812 16:37:01.530416 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:01.530418 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:01.530431 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:01.530520 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:01.531164 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:01.531907 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:01.532673 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:01.532987 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:01.533041 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:01.535536 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:01.535550 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:01.535555 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:01.535563 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:01.537066 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:01.537467 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:01.537478 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:01.537479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:01.537539 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:01.537758 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:01.537759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:01.537760 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:01.537907 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:01.538093 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:01.538139 (+ 46us) server_negotiation.cc:299] Negotiation successful
0812 16:37:01.538223 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":99,"threads_started":1}
W20260812 16:37:01.538841 1360 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59630
Aug 12 16:37:01 dist-test-slave-w4v5 krb5kdc[1029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 16:37:01 dist-test-slave-w4v5 krb5kdc[1029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.253.193@KRBTEST.COM
I20260812 16:37:01.563231 2144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:01.555897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59638 (local address 127.0.253.193:33125)
0812 16:37:01.555972 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:01.555975 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:01.556057 (+ 82us) server_negotiation.cc:407] Connection header received
0812 16:37:01.556159 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:01.556160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:01.556175 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:01.556252 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:01.556842 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:01.557349 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:01.558133 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:01.558349 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:01.558427 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:01.560702 (+ 2275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:01.560704 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:01.560704 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:01.560719 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:01.561983 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:01.562381 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:01.562383 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:01.562383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:01.562448 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:01.562674 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:01.562675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:01.562675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:01.562781 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:01.562935 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:01.562998 (+ 63us) server_negotiation.cc:299] Negotiation successful
0812 16:37:01.563122 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 16:37:01.563881 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 1262
I20260812 16:37:01.582870 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 1546
I20260812 16:37:01.588136 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 1847
I20260812 16:37:01.604151 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 1062
2026-08-12T16:37:01Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2307 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:37:01 dist-test-slave-w4v5 krb5kdc[2156](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:37:01 dist-test-slave-w4v5 krb5kdc[2156](info): set up 2 sockets
Aug 12 16:37:01 dist-test-slave-w4v5 krb5kdc[2156](info): commencing operation
krb5kdc: starting...
W20260812 16:37:02.927901 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.297s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:37:03 dist-test-slave-w4v5 krb5kdc[2156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:37:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:37:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
I20260812 16:37:03.054613 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:37677
--webserver_interface=127.0.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:46057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:37677
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:03.141930 2179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:03.142139 2179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:03.142163 2179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:03.143384 2179 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:37:03.143430 2179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:03.143445 2179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:03.143458 2179 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:37:03.143518 2179 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:37:03.145056 2179 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:46057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:37677
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:37677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:03.145336 2179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:03.145576 2179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:03.150822 2187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:03.150899 2192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:03.151001 2179 server_base.cc:1034] running on GCE node
W20260812 16:37:03.151023 2189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:03.151386 2179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:03.151678 2179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:03.152830 2179 hybrid_clock.cc:648] HybridClock initialized: now 1786552623152809 us; error 38 us; skew 500 ppm
Aug 12 16:37:03 dist-test-slave-w4v5 krb5kdc[2156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552623, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:03.155490 2179 init.cc:377] Logged in from keytab as kudu/127.0.253.254@KRBTEST.COM (short username kudu)
I20260812 16:37:03.155929 2179 webserver.cc:460] Webserver started at http://127.0.253.254:36947/ using document root <none> and password file <none>
I20260812 16:37:03.156164 2179 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:03.156242 2179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:03.156399 2179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:03.157485 2179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "865c6f7be7d94be3909d6ed70f9814dc"
format_stamp: "Formatted at 2026-08-12 16:37:03 on dist-test-slave-w4v5"
server_key: "3d5e37e486186aad2f072e7872984f9e"
server_key_iv: "b8059303980bc045fe65ca610ba39b6e"
server_key_version: "encryptionkey@0"
I20260812 16:37:03.157861 2179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "865c6f7be7d94be3909d6ed70f9814dc"
format_stamp: "Formatted at 2026-08-12 16:37:03 on dist-test-slave-w4v5"
server_key: "3d5e37e486186aad2f072e7872984f9e"
server_key_iv: "b8059303980bc045fe65ca610ba39b6e"
server_key_version: "encryptionkey@0"
I20260812 16:37:03.159301 2179 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 16:37:03.160180 2205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:03.160429 2179 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:03.160508 2179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "865c6f7be7d94be3909d6ed70f9814dc"
format_stamp: "Formatted at 2026-08-12 16:37:03 on dist-test-slave-w4v5"
server_key: "3d5e37e486186aad2f072e7872984f9e"
server_key_iv: "b8059303980bc045fe65ca610ba39b6e"
server_key_version: "encryptionkey@0"
I20260812 16:37:03.160558 2179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:03.171456 2179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:03.173427 2179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:03.173539 2179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:03.177500 2179 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:37677
I20260812 16:37:03.177826 2308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:37677 every 8 connection(s)
I20260812 16:37:03.177896 2179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:37:03.178615 2309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:03.180661 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 2179
I20260812 16:37:03.180769 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:37:03.180969 1015 external_mini_cluster.cc:1428] Setting key 17741dceac324087052d045258b265b4
I20260812 16:37:03.181273 2309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 865c6f7be7d94be3909d6ed70f9814dc: Bootstrap starting.
I20260812 16:37:03.182039 2309 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 865c6f7be7d94be3909d6ed70f9814dc: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:03.182354 2309 log.cc:824] T 00000000000000000000000000000000 P 865c6f7be7d94be3909d6ed70f9814dc: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:03.183247 2309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 865c6f7be7d94be3909d6ed70f9814dc: No bootstrap required, opened a new log
I20260812 16:37:03.185745 2309 raft_consensus.cc:348] T 00000000000000000000000000000000 P 865c6f7be7d94be3909d6ed70f9814dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "865c6f7be7d94be3909d6ed70f9814dc" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 37677 } }
I20260812 16:37:03.185890 2309 raft_consensus.cc:374] T 00000000000000000000000000000000 P 865c6f7be7d94be3909d6ed70f9814dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:03.185913 2309 raft_consensus.cc:729] T 00000000000000000000000000000000 P 865c6f7be7d94be3909d6ed70f9814dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 865c6f7be7d94be3909d6ed70f9814dc, State: Initialized, Role: FOLLOWER
I20260812 16:37:03.186094 2309 consensus_queue.cc:260] T 00000000000000000000000000000000 P 865c6f7be7d94be3909d6ed70f9814dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "865c6f7be7d94be3909d6ed70f9814dc" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 37677 } }
I20260812 16:37:03.186200 2309 raft_consensus.cc:388] T 00000000000000000000000000000000 P 865c6f7be7d94be3909d6ed70f9814dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:37:03.186247 2309 raft_consensus.cc:482] T 00000000000000000000000000000000 P 865c6f7be7d94be3909d6ed70f9814dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:37:03.186604 2309 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 865c6f7be7d94be3909d6ed70f9814dc [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:03.187307 2309 raft_consensus.cc:504] T 00000000000000000000000000000000 P 865c6f7be7d94be3909d6ed70f9814dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "865c6f7be7d94be3909d6ed70f9814dc" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 37677 } }
I20260812 16:37:03.187438 2309 leader_election.cc:302] T 00000000000000000000000000000000 P 865c6f7be7d94be3909d6ed70f9814dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 865c6f7be7d94be3909d6ed70f9814dc; no voters:
I20260812 16:37:03.187638 2309 leader_election.cc:288] T 00000000000000000000000000000000 P 865c6f7be7d94be3909d6ed70f9814dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:37:03.187666 2323 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 865c6f7be7d94be3909d6ed70f9814dc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:37:03.187777 2323 raft_consensus.cc:686] T 00000000000000000000000000000000 P 865c6f7be7d94be3909d6ed70f9814dc [term 1 LEADER]: Becoming Leader. State: Replica: 865c6f7be7d94be3909d6ed70f9814dc, State: Running, Role: LEADER
I20260812 16:37:03.187875 2323 consensus_queue.cc:237] T 00000000000000000000000000000000 P 865c6f7be7d94be3909d6ed70f9814dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "865c6f7be7d94be3909d6ed70f9814dc" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 37677 } }
I20260812 16:37:03.187924 2309 sys_catalog.cc:564] T 00000000000000000000000000000000 P 865c6f7be7d94be3909d6ed70f9814dc [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:37:03.188231 2329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 865c6f7be7d94be3909d6ed70f9814dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "865c6f7be7d94be3909d6ed70f9814dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "865c6f7be7d94be3909d6ed70f9814dc" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 37677 } } }
Aug 12 16:37:03 dist-test-slave-w4v5 krb5kdc[2156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:03.188344 2329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 865c6f7be7d94be3909d6ed70f9814dc [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:03.188225 2330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 865c6f7be7d94be3909d6ed70f9814dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 865c6f7be7d94be3909d6ed70f9814dc. Latest consensus state: current_term: 1 leader_uuid: "865c6f7be7d94be3909d6ed70f9814dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "865c6f7be7d94be3909d6ed70f9814dc" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 37677 } } }
I20260812 16:37:03.188699 2330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 865c6f7be7d94be3909d6ed70f9814dc [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:03.188935 2335 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:37:03.189555 2335 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:37:03.191251 2335 catalog_manager.cc:1347] Generated new cluster ID: e05b3bbd52054c21a37f2c4bd6ae5df7
I20260812 16:37:03.191310 2335 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:37:03.192613 2320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:03.181539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59108 (local address 127.0.253.254:37677)
0812 16:37:03.181727 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:03.181737 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:03.181764 (+ 27us) server_negotiation.cc:407] Connection header received
0812 16:37:03.182266 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:03.182274 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:03.182326 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:03.182490 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:03.183275 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.184226 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.185480 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.185930 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.186046 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:03.188908 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:03.188913 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:03.188913 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:03.188927 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:03.190988 (+ 2061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:03.191450 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:03.191453 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:03.191454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:03.191551 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:03.191774 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:03.191775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:03.191775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:03.192010 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:03.192184 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:03.192294 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 16:37:03.192436 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":90,"threads_started":1}
I20260812 16:37:03.198249 2335 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:37:03.198758 2335 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:37:03.203490 2335 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 865c6f7be7d94be3909d6ed70f9814dc: Generated new TSK 0
I20260812 16:37:03.203687 2335 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab.
Entry for principal kudu/127.0.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab.
I20260812 16:37:03.238341 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.193:0
--local_ip_for_outbound_sockets=127.0.253.193
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:37677
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:46057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:03.359225 2368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:03.359505 2368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:03.359540 2368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:03.361241 2368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:03.361302 2368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:03.361341 2368 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.193
I20260812 16:37:03.363583 2368 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:46057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:37677
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.193
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:03.363895 2368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:03.364169 2368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:03.369730 2377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:03.369719 2378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:03.369719 2380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:03.370105 2368 server_base.cc:1034] running on GCE node
I20260812 16:37:03.370324 2368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:03.370604 2368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:03.371802 2368 hybrid_clock.cc:648] HybridClock initialized: now 1786552623371787 us; error 31 us; skew 500 ppm
Aug 12 16:37:03 dist-test-slave-w4v5 krb5kdc[2156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552623, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:03.374526 2368 init.cc:377] Logged in from keytab as kudu/127.0.253.193@KRBTEST.COM (short username kudu)
I20260812 16:37:03.375056 2368 webserver.cc:460] Webserver started at http://127.0.253.193:44315/ using document root <none> and password file <none>
I20260812 16:37:03.375329 2368 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:03.375397 2368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:03.375535 2368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:03.376808 2368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-0/data/instance:
uuid: "6265a194f79b4cb9924439a0cd3250f4"
format_stamp: "Formatted at 2026-08-12 16:37:03 on dist-test-slave-w4v5"
server_key: "c940fa6c070adb7180a77e8a2f31b1ab"
server_key_iv: "55cc46cf1e19c6ee665cd81d4cef75ee"
server_key_version: "encryptionkey@0"
I20260812 16:37:03.377213 2368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance:
uuid: "6265a194f79b4cb9924439a0cd3250f4"
format_stamp: "Formatted at 2026-08-12 16:37:03 on dist-test-slave-w4v5"
server_key: "c940fa6c070adb7180a77e8a2f31b1ab"
server_key_iv: "55cc46cf1e19c6ee665cd81d4cef75ee"
server_key_version: "encryptionkey@0"
I20260812 16:37:03.378608 2368 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:37:03.379464 2393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:03.379654 2368 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 16:37:03.379769 2368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-0/wal
uuid: "6265a194f79b4cb9924439a0cd3250f4"
format_stamp: "Formatted at 2026-08-12 16:37:03 on dist-test-slave-w4v5"
server_key: "c940fa6c070adb7180a77e8a2f31b1ab"
server_key_iv: "55cc46cf1e19c6ee665cd81d4cef75ee"
server_key_version: "encryptionkey@0"
I20260812 16:37:03.379849 2368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:03.392923 2368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:03.394800 2368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:03.394965 2368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:03.395316 2368 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:37:03.395656 2368 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:03.395713 2368 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:03.395756 2368 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:03.395782 2368 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:03.401580 2368 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.193:39981
I20260812 16:37:03.401661 2619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.193:39981 every 8 connection(s)
I20260812 16:37:03.401999 2368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
I20260812 16:37:03.405448 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 2368
I20260812 16:37:03.405572 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance
I20260812 16:37:03.405750 1015 external_mini_cluster.cc:1428] Setting key e36ad0462d20f15baa8d54a0051b9b81
Aug 12 16:37:03 dist-test-slave-w4v5 krb5kdc[2156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552623, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:03.412245 2320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:03.402739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:37787 (local address 127.0.253.254:37677)
0812 16:37:03.402824 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:03.402827 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:03.402995 (+ 168us) server_negotiation.cc:407] Connection header received
0812 16:37:03.404060 (+ 1065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:03.404061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:03.404071 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:03.404127 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:03.404724 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.405231 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.406326 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.406617 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.406689 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:03.409113 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:03.409114 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:03.409115 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:03.409123 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:03.410557 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:03.411205 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:03.411207 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:03.411207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:03.411262 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:03.411544 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:03.411545 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:03.411545 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:03.411671 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:03.411751 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:03.412043 (+ 292us) server_negotiation.cc:299] Negotiation successful
0812 16:37:03.412141 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 16:37:03.412434 2630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:03.402688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:37677 (local address 127.0.253.193:37787)
0812 16:37:03.402866 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:03.402902 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:03.403960 (+ 1058us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:03.404181 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:03.404185 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:03.404460 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:03.404645 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:03.404649 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.405312 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.405313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:03.406254 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:03.406257 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.406720 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.406721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:03.406824 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:03.407380 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:03.407389 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:03.409010 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:03.410704 (+ 1694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:03.410711 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:03.410714 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:03.411042 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:03.411346 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:03.411347 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:03.411347 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:03.411467 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:03.411740 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:03.411749 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:03.411893 (+ 144us) client_negotiation.cc:769] Sending connection context
0812 16:37:03.412008 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 16:37:03.412174 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":93,"threads_started":1}
I20260812 16:37:03.412931 2626 heartbeater.cc:347] Connected to a master server at 127.0.253.254:37677
I20260812 16:37:03.413053 2626 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:03.413264 2626 heartbeater.cc:511] Master 127.0.253.254:37677 requested a full tablet report, sending...
I20260812 16:37:03.413796 2242 ts_manager.cc:194] Registered new tserver with Master: 6265a194f79b4cb9924439a0cd3250f4 (127.0.253.193:39981)
I20260812 16:37:03.414443 2242 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.193@KRBTEST.COM'} at 127.0.253.193:37787
WARNING: no policy specified for kudu/127.0.253.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab.
Entry for principal kudu/127.0.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab.
I20260812 16:37:03.432080 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.194:0
--local_ip_for_outbound_sockets=127.0.253.194
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:37677
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:46057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:03.515290 2637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:03.515517 2637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:03.515542 2637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:03.516647 2637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:03.516680 2637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:03.516707 2637 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.194
I20260812 16:37:03.518196 2637 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:46057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:37677
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.194
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:03.518386 2637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:03.518574 2637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:03.523820 2660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:03.524050 2656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:03.523869 2654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:03.524063 2637 server_base.cc:1034] running on GCE node
I20260812 16:37:03.524351 2637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:03.524612 2637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:03.525807 2637 hybrid_clock.cc:648] HybridClock initialized: now 1786552623525799 us; error 54 us; skew 500 ppm
Aug 12 16:37:03 dist-test-slave-w4v5 krb5kdc[2156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552623, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:03.528689 2637 init.cc:377] Logged in from keytab as kudu/127.0.253.194@KRBTEST.COM (short username kudu)
I20260812 16:37:03.529176 2637 webserver.cc:460] Webserver started at http://127.0.253.194:36573/ using document root <none> and password file <none>
I20260812 16:37:03.529413 2637 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:03.529485 2637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:03.529644 2637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:03.530759 2637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-1/data/instance:
uuid: "6e4bd95848e14c4ab48e1a8b1b055e65"
format_stamp: "Formatted at 2026-08-12 16:37:03 on dist-test-slave-w4v5"
server_key: "837cb52e785fa000bfb06e96254c2a41"
server_key_iv: "36019d8f78bfebdea428e83b000a629c"
server_key_version: "encryptionkey@0"
I20260812 16:37:03.531219 2637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance:
uuid: "6e4bd95848e14c4ab48e1a8b1b055e65"
format_stamp: "Formatted at 2026-08-12 16:37:03 on dist-test-slave-w4v5"
server_key: "837cb52e785fa000bfb06e96254c2a41"
server_key_iv: "36019d8f78bfebdea428e83b000a629c"
server_key_version: "encryptionkey@0"
I20260812 16:37:03.532589 2637 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:03.533427 2686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:03.533612 2637 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:03.533720 2637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-1/wal
uuid: "6e4bd95848e14c4ab48e1a8b1b055e65"
format_stamp: "Formatted at 2026-08-12 16:37:03 on dist-test-slave-w4v5"
server_key: "837cb52e785fa000bfb06e96254c2a41"
server_key_iv: "36019d8f78bfebdea428e83b000a629c"
server_key_version: "encryptionkey@0"
I20260812 16:37:03.533807 2637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:03.560750 2637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:03.563321 2637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:03.563498 2637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:03.563750 2637 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:37:03.564110 2637 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:03.564167 2637 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:03.564213 2637 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:03.564241 2637 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:03.570010 2637 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.194:44209
I20260812 16:37:03.570138 2916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.194:44209 every 8 connection(s)
I20260812 16:37:03.570425 2637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
Aug 12 16:37:03 dist-test-slave-w4v5 krb5kdc[2156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552623, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:03.577468 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 2637
I20260812 16:37:03.577600 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance
I20260812 16:37:03.577796 1015 external_mini_cluster.cc:1428] Setting key a9569f0452758a2a959a44bc0f66006b
I20260812 16:37:03.580616 2320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:03.571224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:59375 (local address 127.0.253.254:37677)
0812 16:37:03.571270 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:03.571272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:03.571529 (+ 257us) server_negotiation.cc:407] Connection header received
0812 16:37:03.572605 (+ 1076us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:03.572607 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:03.572620 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:03.572677 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:03.573316 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.573825 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.574880 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.575137 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.575224 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:03.577295 (+ 2071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:03.577297 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:03.577297 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:03.577307 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:03.579006 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:03.579541 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:03.579542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:03.579542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:03.579597 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:03.579887 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:03.579888 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:03.579888 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:03.579994 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:03.580047 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:03.580385 (+ 338us) server_negotiation.cc:299] Negotiation successful
0812 16:37:03.580489 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 16:37:03.580816 2925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:03.571227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:37677 (local address 127.0.253.194:59375)
0812 16:37:03.571406 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:03.571443 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:03.572464 (+ 1021us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:03.572756 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:03.572759 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:03.573012 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:03.573215 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:03.573219 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.573924 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.573925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:03.574769 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:03.574771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.575235 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.575236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:03.575367 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:03.575796 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:03.575804 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:03.577186 (+ 1382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:03.579117 (+ 1931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:03.579125 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:03.579127 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:03.579464 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:03.579668 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:03.579669 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:03.579669 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:03.579821 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:03.580066 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:03.580071 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:03.580247 (+ 176us) client_negotiation.cc:769] Sending connection context
0812 16:37:03.580365 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 16:37:03.580556 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":86,"threads_started":1}
I20260812 16:37:03.581283 2923 heartbeater.cc:347] Connected to a master server at 127.0.253.254:37677
I20260812 16:37:03.581382 2923 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:03.581604 2923 heartbeater.cc:511] Master 127.0.253.254:37677 requested a full tablet report, sending...
I20260812 16:37:03.582042 2242 ts_manager.cc:194] Registered new tserver with Master: 6e4bd95848e14c4ab48e1a8b1b055e65 (127.0.253.194:44209)
I20260812 16:37:03.582602 2242 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.194@KRBTEST.COM'} at 127.0.253.194:59375
WARNING: no policy specified for kudu/127.0.253.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab.
Entry for principal kudu/127.0.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab.
I20260812 16:37:03.606578 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.195:0
--local_ip_for_outbound_sockets=127.0.253.195
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:37677
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:46057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:03.691967 2931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:03.692189 2931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:03.692214 2931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:03.693397 2931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:03.693441 2931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:03.693468 2931 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.195
I20260812 16:37:03.695019 2931 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:46057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:37677
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.195
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:03.695348 2931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:03.695613 2931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:03.701220 2942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:03.701270 2946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:03.701220 2941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:03.701383 2931 server_base.cc:1034] running on GCE node
I20260812 16:37:03.701627 2931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:03.701864 2931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:03.703059 2931 hybrid_clock.cc:648] HybridClock initialized: now 1786552623703049 us; error 32 us; skew 500 ppm
Aug 12 16:37:03 dist-test-slave-w4v5 krb5kdc[2156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552623, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:03.706063 2931 init.cc:377] Logged in from keytab as kudu/127.0.253.195@KRBTEST.COM (short username kudu)
I20260812 16:37:03.706574 2931 webserver.cc:460] Webserver started at http://127.0.253.195:44911/ using document root <none> and password file <none>
I20260812 16:37:03.706840 2931 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:03.706912 2931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:03.707115 2931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:03.708196 2931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-2/data/instance:
uuid: "340fa409d6674e12ba010c6ae2bdaddc"
format_stamp: "Formatted at 2026-08-12 16:37:03 on dist-test-slave-w4v5"
server_key: "280acaf386a41369a2813d7d6029c90c"
server_key_iv: "aca859c829b669115987b2ad428e4b63"
server_key_version: "encryptionkey@0"
I20260812 16:37:03.708580 2931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance:
uuid: "340fa409d6674e12ba010c6ae2bdaddc"
format_stamp: "Formatted at 2026-08-12 16:37:03 on dist-test-slave-w4v5"
server_key: "280acaf386a41369a2813d7d6029c90c"
server_key_iv: "aca859c829b669115987b2ad428e4b63"
server_key_version: "encryptionkey@0"
I20260812 16:37:03.710059 2931 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 16:37:03.710909 2972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:03.711139 2931 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:03.711263 2931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-2/wal
uuid: "340fa409d6674e12ba010c6ae2bdaddc"
format_stamp: "Formatted at 2026-08-12 16:37:03 on dist-test-slave-w4v5"
server_key: "280acaf386a41369a2813d7d6029c90c"
server_key_iv: "aca859c829b669115987b2ad428e4b63"
server_key_version: "encryptionkey@0"
I20260812 16:37:03.711341 2931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:03.723188 2931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:03.725371 2931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:03.725485 2931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:03.725684 2931 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:37:03.726008 2931 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:03.726044 2931 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:03.726091 2931 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:03.726117 2931 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:03.732244 2931 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.195:43041
I20260812 16:37:03.732288 3196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.195:43041 every 8 connection(s)
I20260812 16:37:03.732632 2931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
Aug 12 16:37:03 dist-test-slave-w4v5 krb5kdc[2156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552623, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:03.741724 2320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:03.733393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:55761 (local address 127.0.253.254:37677)
0812 16:37:03.733484 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:03.733486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:03.733627 (+ 141us) server_negotiation.cc:407] Connection header received
0812 16:37:03.734547 (+ 920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:03.734548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:03.734558 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:03.734611 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:03.735270 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.735791 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.736864 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.737056 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.737112 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:03.739284 (+ 2172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:03.739286 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:03.739286 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:03.739293 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:03.740528 (+ 1235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:03.740925 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:03.740926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:03.740926 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:03.740978 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:03.741192 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:03.741193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:03.741193 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:03.741298 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:03.741348 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:03.741582 (+ 234us) server_negotiation.cc:299] Negotiation successful
0812 16:37:03.741656 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 16:37:03.741911 3203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:03.733333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:37677 (local address 127.0.253.195:55761)
0812 16:37:03.733498 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:03.733535 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:03.734455 (+ 920us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:03.734674 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:03.734677 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:03.734931 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:03.735194 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:03.735199 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.735885 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.735886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:03.736781 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:03.736784 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.737132 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.737133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:03.737240 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:03.737694 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:03.737702 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:03.739174 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:03.740602 (+ 1428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:03.740608 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:03.740610 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:03.740857 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:03.741037 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:03.741038 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:03.741038 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:03.741138 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:03.741353 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:03.741357 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:03.741473 (+ 116us) client_negotiation.cc:769] Sending connection context
0812 16:37:03.741553 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 16:37:03.741675 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":103,"threads_started":1}
I20260812 16:37:03.741997 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 2931
I20260812 16:37:03.742089 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance
I20260812 16:37:03.742295 1015 external_mini_cluster.cc:1428] Setting key 0220e0d9ac8e394388ab17574a03e326
I20260812 16:37:03.742443 3198 heartbeater.cc:347] Connected to a master server at 127.0.253.254:37677
I20260812 16:37:03.742558 3198 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:03.742800 3198 heartbeater.cc:511] Master 127.0.253.254:37677 requested a full tablet report, sending...
I20260812 16:37:03.743214 2242 ts_manager.cc:194] Registered new tserver with Master: 340fa409d6674e12ba010c6ae2bdaddc (127.0.253.195:43041)
I20260812 16:37:03.743608 2242 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.195@KRBTEST.COM'} at 127.0.253.195:55761
I20260812 16:37:03.754273 1015 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:37:03.763720 2320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:03.756390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59124 (local address 127.0.253.254:37677)
0812 16:37:03.756459 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:03.756462 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:03.756604 (+ 142us) server_negotiation.cc:407] Connection header received
0812 16:37:03.756695 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:03.756696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:03.756710 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:03.756769 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:03.757390 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.757897 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.758661 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.758857 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.758917 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:03.760429 (+ 1512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:03.760431 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:03.760431 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:03.760437 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:03.762153 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:03.762578 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:03.762580 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:03.762580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:03.762658 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:03.762944 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:03.762945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:03.762946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:03.763166 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:03.763292 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:03.763490 (+ 198us) server_negotiation.cc:299] Negotiation successful
0812 16:37:03.763617 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 16:37:03.767227 2242 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59124:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:37:03.769145 2242 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:37:03.776813 3224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:03.772443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.195:43041 (local address 127.0.0.1:57196)
0812 16:37:03.772581 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:03.772623 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:03.772687 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:03.773063 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:03.773067 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:03.773069 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:03.773154 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:03.773156 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.774337 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.774339 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:03.775641 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:03.775644 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.776479 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.776480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:03.776567 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:03.776585 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 16:37:03.776664 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 16:37:03.776728 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":25,"thread_start_us":89,"threads_started":1}
I20260812 16:37:03.776821 3227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:03.772571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57196 (local address 127.0.253.195:43041)
0812 16:37:03.772858 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:03.772864 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:03.772877 (+ 13us) server_negotiation.cc:407] Connection header received
0812 16:37:03.772923 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:03.772925 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:03.772943 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:03.773046 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:03.773269 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.774231 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.775740 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.776380 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.776467 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:03.776535 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:03.776695 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 16:37:03.776747 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":122,"threads_started":1}
I20260812 16:37:03.777242 3228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:03.772690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:44209 (local address 127.0.0.1:58594)
0812 16:37:03.772868 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:03.772878 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:03.772922 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:03.773191 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:03.773192 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:03.773195 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:03.773246 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:03.773248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.774494 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.774495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:03.775989 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:03.775992 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.776973 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.776974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:03.777046 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:03.777051 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:37:03.777101 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 16:37:03.777147 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":7,"thread_start_us":137,"threads_started":1}
I20260812 16:37:03.777242 3230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:03.772935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:39981 (local address 127.0.0.1:54218)
0812 16:37:03.773401 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:03.773412 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:03.773459 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:03.773812 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:03.773813 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:03.773815 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:03.773852 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:03.773854 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.774926 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.774927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:03.775891 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:03.775893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.777000 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.777000 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:03.777066 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:03.777069 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:37:03.777125 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 16:37:03.777155 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"mutex_wait_us":1,"thread_start_us":92,"threads_started":1}
I20260812 16:37:03.777303 3231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:03.773193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54218 (local address 127.0.253.193:39981)
0812 16:37:03.773576 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:03.773581 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:03.773591 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:37:03.773629 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:03.773631 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:03.773645 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:03.773725 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:03.773942 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.774824 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.775979 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.776732 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.776808 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:03.776870 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:03.777162 (+ 292us) server_negotiation.cc:299] Negotiation successful
0812 16:37:03.777220 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":92,"threads_started":1}
I20260812 16:37:03.777518 3229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:03.772822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58594 (local address 127.0.253.194:44209)
0812 16:37:03.773024 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:03.773028 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:03.773038 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:37:03.773077 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:03.773079 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:03.773093 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:03.773190 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:03.773316 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.774388 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.776053 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.776821 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.776883 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:03.776935 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:03.777382 (+ 447us) server_negotiation.cc:299] Negotiation successful
0812 16:37:03.777435 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":91,"threads_started":1}
I20260812 16:37:03.778015 2489 tablet_service.cc:1467] Processing CreateTablet for tablet c9849d55918c45a9bdca2e3ef070a749 (DEFAULT_TABLE table=test-table [id=daf635350325418c98ba86f8b11cd880]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:03.778033 3057 tablet_service.cc:1467] Processing CreateTablet for tablet c9849d55918c45a9bdca2e3ef070a749 (DEFAULT_TABLE table=test-table [id=daf635350325418c98ba86f8b11cd880]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:03.778167 2789 tablet_service.cc:1467] Processing CreateTablet for tablet c9849d55918c45a9bdca2e3ef070a749 (DEFAULT_TABLE table=test-table [id=daf635350325418c98ba86f8b11cd880]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:03.778326 2489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9849d55918c45a9bdca2e3ef070a749. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:03.778393 3057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9849d55918c45a9bdca2e3ef070a749. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:03.778465 2789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9849d55918c45a9bdca2e3ef070a749. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:03.781432 3232 tablet_bootstrap.cc:492] T c9849d55918c45a9bdca2e3ef070a749 P 6265a194f79b4cb9924439a0cd3250f4: Bootstrap starting.
I20260812 16:37:03.782028 3235 tablet_bootstrap.cc:492] T c9849d55918c45a9bdca2e3ef070a749 P 6e4bd95848e14c4ab48e1a8b1b055e65: Bootstrap starting.
I20260812 16:37:03.782414 3232 tablet_bootstrap.cc:654] T c9849d55918c45a9bdca2e3ef070a749 P 6265a194f79b4cb9924439a0cd3250f4: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:03.782716 3235 tablet_bootstrap.cc:654] T c9849d55918c45a9bdca2e3ef070a749 P 6e4bd95848e14c4ab48e1a8b1b055e65: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:03.783267 3234 tablet_bootstrap.cc:492] T c9849d55918c45a9bdca2e3ef070a749 P 340fa409d6674e12ba010c6ae2bdaddc: Bootstrap starting.
I20260812 16:37:03.783331 3232 log.cc:824] T c9849d55918c45a9bdca2e3ef070a749 P 6265a194f79b4cb9924439a0cd3250f4: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:03.783404 3235 log.cc:824] T c9849d55918c45a9bdca2e3ef070a749 P 6e4bd95848e14c4ab48e1a8b1b055e65: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:03.783951 3234 tablet_bootstrap.cc:654] T c9849d55918c45a9bdca2e3ef070a749 P 340fa409d6674e12ba010c6ae2bdaddc: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:03.784196 3232 tablet_bootstrap.cc:492] T c9849d55918c45a9bdca2e3ef070a749 P 6265a194f79b4cb9924439a0cd3250f4: No bootstrap required, opened a new log
I20260812 16:37:03.784225 3235 tablet_bootstrap.cc:492] T c9849d55918c45a9bdca2e3ef070a749 P 6e4bd95848e14c4ab48e1a8b1b055e65: No bootstrap required, opened a new log
I20260812 16:37:03.784281 3232 ts_tablet_manager.cc:1397] T c9849d55918c45a9bdca2e3ef070a749 P 6265a194f79b4cb9924439a0cd3250f4: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 16:37:03.784297 3235 ts_tablet_manager.cc:1397] T c9849d55918c45a9bdca2e3ef070a749 P 6e4bd95848e14c4ab48e1a8b1b055e65: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:37:03.784565 3234 log.cc:824] T c9849d55918c45a9bdca2e3ef070a749 P 340fa409d6674e12ba010c6ae2bdaddc: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:03.785897 3235 raft_consensus.cc:348] T c9849d55918c45a9bdca2e3ef070a749 P 6e4bd95848e14c4ab48e1a8b1b055e65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "340fa409d6674e12ba010c6ae2bdaddc" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43041 } } peers { permanent_uuid: "6e4bd95848e14c4ab48e1a8b1b055e65" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44209 } } peers { permanent_uuid: "6265a194f79b4cb9924439a0cd3250f4" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 39981 } }
I20260812 16:37:03.785910 3232 raft_consensus.cc:348] T c9849d55918c45a9bdca2e3ef070a749 P 6265a194f79b4cb9924439a0cd3250f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "340fa409d6674e12ba010c6ae2bdaddc" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43041 } } peers { permanent_uuid: "6e4bd95848e14c4ab48e1a8b1b055e65" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44209 } } peers { permanent_uuid: "6265a194f79b4cb9924439a0cd3250f4" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 39981 } }
I20260812 16:37:03.786041 3235 raft_consensus.cc:374] T c9849d55918c45a9bdca2e3ef070a749 P 6e4bd95848e14c4ab48e1a8b1b055e65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:03.786063 3235 raft_consensus.cc:729] T c9849d55918c45a9bdca2e3ef070a749 P 6e4bd95848e14c4ab48e1a8b1b055e65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e4bd95848e14c4ab48e1a8b1b055e65, State: Initialized, Role: FOLLOWER
I20260812 16:37:03.786065 3232 raft_consensus.cc:374] T c9849d55918c45a9bdca2e3ef070a749 P 6265a194f79b4cb9924439a0cd3250f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:03.786108 3232 raft_consensus.cc:729] T c9849d55918c45a9bdca2e3ef070a749 P 6265a194f79b4cb9924439a0cd3250f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6265a194f79b4cb9924439a0cd3250f4, State: Initialized, Role: FOLLOWER
I20260812 16:37:03.786219 3235 consensus_queue.cc:260] T c9849d55918c45a9bdca2e3ef070a749 P 6e4bd95848e14c4ab48e1a8b1b055e65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "340fa409d6674e12ba010c6ae2bdaddc" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43041 } } peers { permanent_uuid: "6e4bd95848e14c4ab48e1a8b1b055e65" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44209 } } peers { permanent_uuid: "6265a194f79b4cb9924439a0cd3250f4" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 39981 } }
I20260812 16:37:03.786221 3232 consensus_queue.cc:260] T c9849d55918c45a9bdca2e3ef070a749 P 6265a194f79b4cb9924439a0cd3250f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "340fa409d6674e12ba010c6ae2bdaddc" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43041 } } peers { permanent_uuid: "6e4bd95848e14c4ab48e1a8b1b055e65" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44209 } } peers { permanent_uuid: "6265a194f79b4cb9924439a0cd3250f4" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 39981 } }
I20260812 16:37:03.786512 3235 ts_tablet_manager.cc:1428] T c9849d55918c45a9bdca2e3ef070a749 P 6e4bd95848e14c4ab48e1a8b1b055e65: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 16:37:03.786537 3232 ts_tablet_manager.cc:1428] T c9849d55918c45a9bdca2e3ef070a749 P 6265a194f79b4cb9924439a0cd3250f4: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260812 16:37:03.786640 2923 heartbeater.cc:503] Master 127.0.253.254:37677 was elected leader, sending a full tablet report...
I20260812 16:37:03.786684 2626 heartbeater.cc:503] Master 127.0.253.254:37677 was elected leader, sending a full tablet report...
I20260812 16:37:03.787422 3234 tablet_bootstrap.cc:492] T c9849d55918c45a9bdca2e3ef070a749 P 340fa409d6674e12ba010c6ae2bdaddc: No bootstrap required, opened a new log
I20260812 16:37:03.787514 3234 ts_tablet_manager.cc:1397] T c9849d55918c45a9bdca2e3ef070a749 P 340fa409d6674e12ba010c6ae2bdaddc: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 16:37:03.788910 3234 raft_consensus.cc:348] T c9849d55918c45a9bdca2e3ef070a749 P 340fa409d6674e12ba010c6ae2bdaddc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "340fa409d6674e12ba010c6ae2bdaddc" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43041 } } peers { permanent_uuid: "6e4bd95848e14c4ab48e1a8b1b055e65" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44209 } } peers { permanent_uuid: "6265a194f79b4cb9924439a0cd3250f4" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 39981 } }
I20260812 16:37:03.789054 3234 raft_consensus.cc:374] T c9849d55918c45a9bdca2e3ef070a749 P 340fa409d6674e12ba010c6ae2bdaddc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:03.789076 3234 raft_consensus.cc:729] T c9849d55918c45a9bdca2e3ef070a749 P 340fa409d6674e12ba010c6ae2bdaddc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 340fa409d6674e12ba010c6ae2bdaddc, State: Initialized, Role: FOLLOWER
I20260812 16:37:03.789149 3234 consensus_queue.cc:260] T c9849d55918c45a9bdca2e3ef070a749 P 340fa409d6674e12ba010c6ae2bdaddc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "340fa409d6674e12ba010c6ae2bdaddc" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43041 } } peers { permanent_uuid: "6e4bd95848e14c4ab48e1a8b1b055e65" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44209 } } peers { permanent_uuid: "6265a194f79b4cb9924439a0cd3250f4" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 39981 } }
I20260812 16:37:03.789368 3234 ts_tablet_manager.cc:1428] T c9849d55918c45a9bdca2e3ef070a749 P 340fa409d6674e12ba010c6ae2bdaddc: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:37:03.789906 3198 heartbeater.cc:503] Master 127.0.253.254:37677 was elected leader, sending a full tablet report...
W20260812 16:37:03.821045 2924 tablet.cc:2369] T c9849d55918c45a9bdca2e3ef070a749 P 6e4bd95848e14c4ab48e1a8b1b055e65: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:37:03.841718 3250 raft_consensus.cc:482] T c9849d55918c45a9bdca2e3ef070a749 P 340fa409d6674e12ba010c6ae2bdaddc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:37:03.841859 3250 raft_consensus.cc:504] T c9849d55918c45a9bdca2e3ef070a749 P 340fa409d6674e12ba010c6ae2bdaddc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "340fa409d6674e12ba010c6ae2bdaddc" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43041 } } peers { permanent_uuid: "6e4bd95848e14c4ab48e1a8b1b055e65" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44209 } } peers { permanent_uuid: "6265a194f79b4cb9924439a0cd3250f4" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 39981 } }
I20260812 16:37:03.842502 3250 leader_election.cc:288] T c9849d55918c45a9bdca2e3ef070a749 P 340fa409d6674e12ba010c6ae2bdaddc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e4bd95848e14c4ab48e1a8b1b055e65 (127.0.253.194:44209), 6265a194f79b4cb9924439a0cd3250f4 (127.0.253.193:39981)
I20260812 16:37:03.845495 3229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:03.842810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:38633 (local address 127.0.253.194:44209)
0812 16:37:03.842883 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:03.842885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:03.842897 (+ 12us) server_negotiation.cc:407] Connection header received
0812 16:37:03.842931 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:03.842932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:03.842947 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:03.843062 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:03.843273 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.843779 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.844736 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.845133 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.845224 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:03.845234 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:03.845396 (+ 162us) server_negotiation.cc:299] Negotiation successful
0812 16:37:03.845436 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 16:37:03.845494 3203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:03.842725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:44209 (local address 127.0.253.195:38633)
0812 16:37:03.842791 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:03.842805 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:03.842854 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:03.843094 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:03.843095 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:03.843098 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:03.843178 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:03.843185 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.843867 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.843869 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:03.844631 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:03.844633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.845222 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.845223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:03.845310 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:03.845313 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:37:03.845377 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 16:37:03.845418 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 16:37:03.845980 3231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:03.842985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:39495 (local address 127.0.253.193:39981)
0812 16:37:03.843031 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:03.843034 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:03.843182 (+ 148us) server_negotiation.cc:407] Connection header received
0812 16:37:03.843268 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:03.843275 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:03.843285 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:03.843343 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:03.843530 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.843982 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.844980 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.845370 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.845461 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:03.845470 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:03.845879 (+ 409us) server_negotiation.cc:299] Negotiation successful
0812 16:37:03.845919 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 16:37:03.845985 3251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:03.842848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:39981 (local address 127.0.253.195:39495)
0812 16:37:03.843060 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:03.843113 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:03.843191 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:03.843361 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:03.843362 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:03.843363 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:03.843449 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:03.843450 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.844076 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.844078 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:03.844880 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:03.844883 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.845462 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.845463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:03.845794 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:03.845798 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:37:03.845869 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 16:37:03.845907 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":127,"threads_started":1}
I20260812 16:37:03.846015 2818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9849d55918c45a9bdca2e3ef070a749" candidate_uuid: "340fa409d6674e12ba010c6ae2bdaddc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e4bd95848e14c4ab48e1a8b1b055e65" is_pre_election: true
I20260812 16:37:03.846256 2818 raft_consensus.cc:2454] T c9849d55918c45a9bdca2e3ef070a749 P 6e4bd95848e14c4ab48e1a8b1b055e65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 340fa409d6674e12ba010c6ae2bdaddc in term 0.
I20260812 16:37:03.846379 2534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9849d55918c45a9bdca2e3ef070a749" candidate_uuid: "340fa409d6674e12ba010c6ae2bdaddc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6265a194f79b4cb9924439a0cd3250f4" is_pre_election: true
I20260812 16:37:03.846549 2534 raft_consensus.cc:2454] T c9849d55918c45a9bdca2e3ef070a749 P 6265a194f79b4cb9924439a0cd3250f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 340fa409d6674e12ba010c6ae2bdaddc in term 0.
I20260812 16:37:03.846597 2975 leader_election.cc:302] T c9849d55918c45a9bdca2e3ef070a749 P 340fa409d6674e12ba010c6ae2bdaddc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 340fa409d6674e12ba010c6ae2bdaddc, 6e4bd95848e14c4ab48e1a8b1b055e65; no voters:
I20260812 16:37:03.846849 3250 raft_consensus.cc:2781] T c9849d55918c45a9bdca2e3ef070a749 P 340fa409d6674e12ba010c6ae2bdaddc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:37:03.846915 3250 raft_consensus.cc:482] T c9849d55918c45a9bdca2e3ef070a749 P 340fa409d6674e12ba010c6ae2bdaddc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:37:03.846944 3250 raft_consensus.cc:3037] T c9849d55918c45a9bdca2e3ef070a749 P 340fa409d6674e12ba010c6ae2bdaddc [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:03.847710 3250 raft_consensus.cc:504] T c9849d55918c45a9bdca2e3ef070a749 P 340fa409d6674e12ba010c6ae2bdaddc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "340fa409d6674e12ba010c6ae2bdaddc" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43041 } } peers { permanent_uuid: "6e4bd95848e14c4ab48e1a8b1b055e65" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44209 } } peers { permanent_uuid: "6265a194f79b4cb9924439a0cd3250f4" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 39981 } }
I20260812 16:37:03.847858 3250 leader_election.cc:288] T c9849d55918c45a9bdca2e3ef070a749 P 340fa409d6674e12ba010c6ae2bdaddc [CANDIDATE]: Term 1 election: Requested vote from peers 6e4bd95848e14c4ab48e1a8b1b055e65 (127.0.253.194:44209), 6265a194f79b4cb9924439a0cd3250f4 (127.0.253.193:39981)
I20260812 16:37:03.848080 2534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9849d55918c45a9bdca2e3ef070a749" candidate_uuid: "340fa409d6674e12ba010c6ae2bdaddc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6265a194f79b4cb9924439a0cd3250f4"
I20260812 16:37:03.848073 2818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9849d55918c45a9bdca2e3ef070a749" candidate_uuid: "340fa409d6674e12ba010c6ae2bdaddc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e4bd95848e14c4ab48e1a8b1b055e65"
I20260812 16:37:03.848158 2818 raft_consensus.cc:3037] T c9849d55918c45a9bdca2e3ef070a749 P 6e4bd95848e14c4ab48e1a8b1b055e65 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:03.848158 2534 raft_consensus.cc:3037] T c9849d55918c45a9bdca2e3ef070a749 P 6265a194f79b4cb9924439a0cd3250f4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:03.849097 2818 raft_consensus.cc:2454] T c9849d55918c45a9bdca2e3ef070a749 P 6e4bd95848e14c4ab48e1a8b1b055e65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 340fa409d6674e12ba010c6ae2bdaddc in term 1.
I20260812 16:37:03.849097 2534 raft_consensus.cc:2454] T c9849d55918c45a9bdca2e3ef070a749 P 6265a194f79b4cb9924439a0cd3250f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 340fa409d6674e12ba010c6ae2bdaddc in term 1.
I20260812 16:37:03.849295 2975 leader_election.cc:302] T c9849d55918c45a9bdca2e3ef070a749 P 340fa409d6674e12ba010c6ae2bdaddc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 340fa409d6674e12ba010c6ae2bdaddc, 6265a194f79b4cb9924439a0cd3250f4; no voters:
I20260812 16:37:03.849390 3250 raft_consensus.cc:2781] T c9849d55918c45a9bdca2e3ef070a749 P 340fa409d6674e12ba010c6ae2bdaddc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:37:03.849592 3250 raft_consensus.cc:686] T c9849d55918c45a9bdca2e3ef070a749 P 340fa409d6674e12ba010c6ae2bdaddc [term 1 LEADER]: Becoming Leader. State: Replica: 340fa409d6674e12ba010c6ae2bdaddc, State: Running, Role: LEADER
I20260812 16:37:03.849695 3250 consensus_queue.cc:237] T c9849d55918c45a9bdca2e3ef070a749 P 340fa409d6674e12ba010c6ae2bdaddc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "340fa409d6674e12ba010c6ae2bdaddc" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43041 } } peers { permanent_uuid: "6e4bd95848e14c4ab48e1a8b1b055e65" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44209 } } peers { permanent_uuid: "6265a194f79b4cb9924439a0cd3250f4" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 39981 } }
I20260812 16:37:03.850679 2241 catalog_manager.cc:5495] T c9849d55918c45a9bdca2e3ef070a749 P 340fa409d6674e12ba010c6ae2bdaddc reported cstate change: term changed from 0 to 1, leader changed from <none> to 340fa409d6674e12ba010c6ae2bdaddc (127.0.253.195). New cstate: current_term: 1 leader_uuid: "340fa409d6674e12ba010c6ae2bdaddc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "340fa409d6674e12ba010c6ae2bdaddc" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43041 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e4bd95848e14c4ab48e1a8b1b055e65" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6265a194f79b4cb9924439a0cd3250f4" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 39981 } health_report { overall_health: UNKNOWN } } }
Aug 12 16:37:03 dist-test-slave-w4v5 krb5kdc[2156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.253.193@KRBTEST.COM
I20260812 16:37:03.877357 3231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:03.869836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54228 (local address 127.0.253.193:39981)
0812 16:37:03.869934 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:03.869936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:03.870027 (+ 91us) server_negotiation.cc:407] Connection header received
0812 16:37:03.870118 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:03.870119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:03.870130 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:03.870197 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:03.870839 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.871389 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.872146 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.872350 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.872429 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:03.874809 (+ 2380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:03.874822 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:03.874822 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:03.874829 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:03.876324 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:03.876644 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:03.876647 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:03.876647 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:03.876702 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:03.876874 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:03.876874 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:03.876875 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:03.877012 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:03.877116 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:03.877173 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 16:37:03.877248 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
Aug 12 16:37:03 dist-test-slave-w4v5 krb5kdc[2156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552623, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 16:37:03 dist-test-slave-w4v5 krb5kdc[2156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552623, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.253.193@KRBTEST.COM
I20260812 16:37:03.902920 3231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:03.895323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54230 (local address 127.0.253.193:39981)
0812 16:37:03.895395 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:03.895397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:03.895488 (+ 91us) server_negotiation.cc:407] Connection header received
0812 16:37:03.895594 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:03.895595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:03.895606 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:03.895660 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:03.896251 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.896738 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.897614 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.897828 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.897914 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:03.900170 (+ 2256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:03.900172 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:03.900172 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:03.900181 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:03.901554 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:03.901958 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:03.901960 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:03.901960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:03.902026 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:03.902246 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:03.902247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:03.902247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:03.902355 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:03.902464 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:03.902711 (+ 247us) server_negotiation.cc:299] Negotiation successful
0812 16:37:03.902795 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260812 16:37:03.902804 2627 tablet.cc:2369] T c9849d55918c45a9bdca2e3ef070a749 P 6265a194f79b4cb9924439a0cd3250f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 16:37:03 dist-test-slave-w4v5 krb5kdc[2156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 16:37:03 dist-test-slave-w4v5 krb5kdc[2156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.253.193@KRBTEST.COM
I20260812 16:37:03.927512 3231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:03.920567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54244 (local address 127.0.253.193:39981)
0812 16:37:03.920642 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:03.920644 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:03.920748 (+ 104us) server_negotiation.cc:407] Connection header received
0812 16:37:03.920846 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:03.920847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:03.920859 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:03.920935 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:03.921519 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.922009 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.922768 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:03.922963 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:03.923044 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:03.925333 (+ 2289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:03.925335 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:03.925335 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:03.925342 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:03.926517 (+ 1175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:03.926834 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:03.926836 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:03.926836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:03.926887 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:03.927048 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:03.927048 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:03.927048 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:03.927201 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:03.927285 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:03.927382 (+ 97us) server_negotiation.cc:299] Negotiation successful
0812 16:37:03.927443 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260812 16:37:03.928269 2448 tablet_service.cc:2908] Rejecting scan request for tablet c9849d55918c45a9bdca2e3ef070a749: Uninitialized: clean time has not yet been initialized
I20260812 16:37:03.929288 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 2368
I20260812 16:37:03.940474 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 2637
I20260812 16:37:03.956207 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 2931
I20260812 16:37:03.972040 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 2179
2026-08-12T16:37:03Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2368 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:37:04 dist-test-slave-w4v5 krb5kdc[3280](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:37:04 dist-test-slave-w4v5 krb5kdc[3280](info): set up 2 sockets
Aug 12 16:37:04 dist-test-slave-w4v5 krb5kdc[3280](info): commencing operation
krb5kdc: starting...
W20260812 16:37:05.284200 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.285s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:37:05 dist-test-slave-w4v5 krb5kdc[3280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:37:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:37:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
I20260812 16:37:05.411753 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:37355
--webserver_interface=127.0.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:35659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:37355
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:05.494949 3306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:05.495241 3306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:05.495283 3306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:05.496467 3306 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:37:05.496546 3306 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 16:37:05.496568 3306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:05.496598 3306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:05.496646 3306 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:37:05.496675 3306 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:37:05.498188 3306 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:35659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:37355
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:37355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:05.498474 3306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:05.498726 3306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:05.503844 3318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:05.503840 3319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:05.503849 3324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:05.504274 3306 server_base.cc:1034] running on GCE node
I20260812 16:37:05.504566 3306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:05.504889 3306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:05.506063 3306 hybrid_clock.cc:648] HybridClock initialized: now 1786552625506041 us; error 40 us; skew 500 ppm
Aug 12 16:37:05 dist-test-slave-w4v5 krb5kdc[3280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552625, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:05.508697 3306 init.cc:377] Logged in from keytab as kudu/127.0.253.254@KRBTEST.COM (short username kudu)
I20260812 16:37:05.509089 3306 webserver.cc:460] Webserver started at http://127.0.253.254:38385/ using document root <none> and password file <none>
I20260812 16:37:05.509327 3306 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:05.509374 3306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:05.509526 3306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:05.510634 3306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "80e95251a626458bb717456269c4ea24"
format_stamp: "Formatted at 2026-08-12 16:37:05 on dist-test-slave-w4v5"
server_key: "c91b3ded105db51df5661b095d6670ca"
server_key_iv: "77b08a7b99bd622f3a626d828f7bae99"
server_key_version: "encryptionkey@0"
I20260812 16:37:05.511029 3306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "80e95251a626458bb717456269c4ea24"
format_stamp: "Formatted at 2026-08-12 16:37:05 on dist-test-slave-w4v5"
server_key: "c91b3ded105db51df5661b095d6670ca"
server_key_iv: "77b08a7b99bd622f3a626d828f7bae99"
server_key_version: "encryptionkey@0"
I20260812 16:37:05.512631 3306 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:37:05.513477 3351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:05.513697 3306 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:05.513806 3306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "80e95251a626458bb717456269c4ea24"
format_stamp: "Formatted at 2026-08-12 16:37:05 on dist-test-slave-w4v5"
server_key: "c91b3ded105db51df5661b095d6670ca"
server_key_iv: "77b08a7b99bd622f3a626d828f7bae99"
server_key_version: "encryptionkey@0"
I20260812 16:37:05.513895 3306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:05.525712 3306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:05.527809 3306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:05.527963 3306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:05.532990 3306 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:37355
I20260812 16:37:05.533080 3451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:37355 every 8 connection(s)
I20260812 16:37:05.533411 3306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:37:05.534163 3453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:05.536590 3453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80e95251a626458bb717456269c4ea24: Bootstrap starting.
I20260812 16:37:05.537037 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 3306
I20260812 16:37:05.537124 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:37:05.537266 3453 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80e95251a626458bb717456269c4ea24: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:05.537303 1015 external_mini_cluster.cc:1428] Setting key e33117c73a779f37df4c3123774c5ae0
I20260812 16:37:05.537575 3453 log.cc:824] T 00000000000000000000000000000000 P 80e95251a626458bb717456269c4ea24: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:05.538371 3453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80e95251a626458bb717456269c4ea24: No bootstrap required, opened a new log
I20260812 16:37:05.539886 3453 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80e95251a626458bb717456269c4ea24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80e95251a626458bb717456269c4ea24" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 37355 } }
I20260812 16:37:05.540019 3453 raft_consensus.cc:374] T 00000000000000000000000000000000 P 80e95251a626458bb717456269c4ea24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:05.540041 3453 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80e95251a626458bb717456269c4ea24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80e95251a626458bb717456269c4ea24, State: Initialized, Role: FOLLOWER
I20260812 16:37:05.540179 3453 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80e95251a626458bb717456269c4ea24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80e95251a626458bb717456269c4ea24" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 37355 } }
I20260812 16:37:05.540259 3453 raft_consensus.cc:388] T 00000000000000000000000000000000 P 80e95251a626458bb717456269c4ea24 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:37:05.540282 3453 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80e95251a626458bb717456269c4ea24 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:37:05.540350 3453 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80e95251a626458bb717456269c4ea24 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:05.540902 3453 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80e95251a626458bb717456269c4ea24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80e95251a626458bb717456269c4ea24" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 37355 } }
I20260812 16:37:05.540992 3453 leader_election.cc:302] T 00000000000000000000000000000000 P 80e95251a626458bb717456269c4ea24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 80e95251a626458bb717456269c4ea24; no voters:
I20260812 16:37:05.541203 3453 leader_election.cc:288] T 00000000000000000000000000000000 P 80e95251a626458bb717456269c4ea24 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:37:05.541236 3463 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80e95251a626458bb717456269c4ea24 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:37:05.541355 3463 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80e95251a626458bb717456269c4ea24 [term 1 LEADER]: Becoming Leader. State: Replica: 80e95251a626458bb717456269c4ea24, State: Running, Role: LEADER
I20260812 16:37:05.541496 3453 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80e95251a626458bb717456269c4ea24 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:37:05.541502 3463 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80e95251a626458bb717456269c4ea24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80e95251a626458bb717456269c4ea24" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 37355 } }
I20260812 16:37:05.542003 3470 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80e95251a626458bb717456269c4ea24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "80e95251a626458bb717456269c4ea24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80e95251a626458bb717456269c4ea24" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 37355 } } }
I20260812 16:37:05.542246 3470 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80e95251a626458bb717456269c4ea24 [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:05.542183 3464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80e95251a626458bb717456269c4ea24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80e95251a626458bb717456269c4ea24. Latest consensus state: current_term: 1 leader_uuid: "80e95251a626458bb717456269c4ea24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80e95251a626458bb717456269c4ea24" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 37355 } } }
I20260812 16:37:05.542500 3464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80e95251a626458bb717456269c4ea24 [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:05.542729 3484 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:37:05.543313 3484 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:37:05.545054 3484 catalog_manager.cc:1347] Generated new cluster ID: 9f61453edec342e299b6f9461ffe55fb
I20260812 16:37:05.545111 3484 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 16:37:05 dist-test-slave-w4v5 krb5kdc[3280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:05.549276 3461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:05.537860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55596 (local address 127.0.253.254:37355)
0812 16:37:05.538045 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:05.538056 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:05.538092 (+ 36us) server_negotiation.cc:407] Connection header received
0812 16:37:05.538552 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:05.538559 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:05.538617 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:05.538764 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:05.539500 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:05.541103 (+ 1603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:05.542047 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:05.542684 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:05.542748 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:05.545817 (+ 3069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:05.545822 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:05.545822 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:05.545835 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:05.547711 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:05.548090 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:05.548094 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:05.548094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:05.548187 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:05.548410 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:05.548411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:05.548411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:05.548658 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:05.548834 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:05.548915 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 16:37:05.549057 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":72,"threads_started":1}
I20260812 16:37:05.556226 3484 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:37:05.556780 3484 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:37:05.569056 3484 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 80e95251a626458bb717456269c4ea24: Generated new TSK 0
I20260812 16:37:05.569281 3484 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab.
Entry for principal kudu/127.0.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab.
I20260812 16:37:05.600307 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.193:0
--local_ip_for_outbound_sockets=127.0.253.193
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:37355
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:35659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:05.684751 3510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:05.684960 3510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:05.684984 3510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:05.686183 3510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:05.686230 3510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:05.686251 3510 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 16:37:05.686268 3510 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.193
I20260812 16:37:05.687666 3510 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:35659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.253.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.253.254:37355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.193
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:05.687852 3510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:05.688035 3510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:05.693405 3526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:05.693416 3523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:05.693420 3522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:05.693751 3510 server_base.cc:1034] running on GCE node
I20260812 16:37:05.693960 3510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:05.694249 3510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:05.695413 3510 hybrid_clock.cc:648] HybridClock initialized: now 1786552625695398 us; error 43 us; skew 500 ppm
Aug 12 16:37:05 dist-test-slave-w4v5 krb5kdc[3280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552625, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:05.697860 3510 init.cc:377] Logged in from keytab as kudu/127.0.253.193@KRBTEST.COM (short username kudu)
I20260812 16:37:05.698299 3510 webserver.cc:460] Webserver started at http://127.0.253.193:44837/ using document root <none> and password file <none>
I20260812 16:37:05.698530 3510 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:05.698601 3510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:05.698758 3510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:05.699875 3510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-0/data/instance:
uuid: "d5d3d26c2a9b4f8c8f839739e0074511"
format_stamp: "Formatted at 2026-08-12 16:37:05 on dist-test-slave-w4v5"
server_key: "010edfb7c31e0c766918be95be7c3674"
server_key_iv: "ba812348326e42529c1cb59b750a7346"
server_key_version: "encryptionkey@0"
I20260812 16:37:05.700258 3510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance:
uuid: "d5d3d26c2a9b4f8c8f839739e0074511"
format_stamp: "Formatted at 2026-08-12 16:37:05 on dist-test-slave-w4v5"
server_key: "010edfb7c31e0c766918be95be7c3674"
server_key_iv: "ba812348326e42529c1cb59b750a7346"
server_key_version: "encryptionkey@0"
I20260812 16:37:05.701644 3510 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 16:37:05.702463 3542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:05.702658 3510 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:37:05.702773 3510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-0/wal
uuid: "d5d3d26c2a9b4f8c8f839739e0074511"
format_stamp: "Formatted at 2026-08-12 16:37:05 on dist-test-slave-w4v5"
server_key: "010edfb7c31e0c766918be95be7c3674"
server_key_iv: "ba812348326e42529c1cb59b750a7346"
server_key_version: "encryptionkey@0"
I20260812 16:37:05.702850 3510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:05.723951 3510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:05.726097 3510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:05.726280 3510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:05.726881 3510 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:05.726941 3510 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:05.726985 3510 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:05.727013 3510 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:05.733820 3510 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.193:38327
I20260812 16:37:05.733898 3771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.193:38327 every 8 connection(s)
I20260812 16:37:05.734192 3510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
Aug 12 16:37:05 dist-test-slave-w4v5 krb5kdc[3280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552625, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:05.736106 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 3510
I20260812 16:37:05.736194 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance
I20260812 16:37:05.736387 1015 external_mini_cluster.cc:1428] Setting key 2b24f59de934265c433294bf94561c5e
I20260812 16:37:05.737797 3461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:05.727581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:58895 (local address 127.0.253.254:37355)
0812 16:37:05.727657 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:05.727659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:05.727914 (+ 255us) server_negotiation.cc:407] Connection header received
0812 16:37:05.728529 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:05.728531 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:05.728544 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:05.728595 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:05.729454 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:05.730088 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:05.731744 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:05.732082 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:05.732157 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:05.734813 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:05.734814 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:05.734815 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:05.734824 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:05.736186 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:05.736693 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:05.736695 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:05.736695 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:05.736768 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:05.737066 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:05.737067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:05.737067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:05.737236 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:05.737318 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:05.737576 (+ 258us) server_negotiation.cc:299] Negotiation successful
0812 16:37:05.737677 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 16:37:05.737815 3560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:05.727558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:37355 (local address 127.0.253.193:58895)
0812 16:37:05.727797 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:05.727838 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:05.728414 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:05.728667 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:05.728671 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:05.729051 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:05.729356 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:05.729363 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:05.730190 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:05.730191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:05.731643 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:05.731647 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:05.732174 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:05.732175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:05.732316 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:05.732908 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:05.732919 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:05.734714 (+ 1795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:05.736283 (+ 1569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:05.736297 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:05.736299 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:05.736593 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:05.736855 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:05.736855 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:05.736856 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:05.736991 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:05.737317 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:05.737323 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:05.737439 (+ 116us) client_negotiation.cc:769] Sending connection context
0812 16:37:05.737525 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 16:37:05.737647 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":171,"threads_started":1}
I20260812 16:37:05.739508 3773 heartbeater.cc:347] Connected to a master server at 127.0.253.254:37355
I20260812 16:37:05.739665 3773 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:05.739876 3773 heartbeater.cc:511] Master 127.0.253.254:37355 requested a full tablet report, sending...
I20260812 16:37:05.740414 3378 ts_manager.cc:194] Registered new tserver with Master: d5d3d26c2a9b4f8c8f839739e0074511 (127.0.253.193:38327)
I20260812 16:37:05.741196 3378 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.193@KRBTEST.COM'} at 127.0.253.193:58895
WARNING: no policy specified for kudu/127.0.253.194@KRBTEST.COM; defaulting to no policy
I20260812 16:37:05.746411 3461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:05.740602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:32921 (local address 127.0.253.254:37355)
0812 16:37:05.740639 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:05.740641 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:05.740696 (+ 55us) server_negotiation.cc:407] Connection header received
0812 16:37:05.740756 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:05.740756 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:05.740767 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:05.740828 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:05.741219 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:05.741874 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:05.742619 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:05.742830 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:05.742918 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:05.743913 (+ 995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:05.743914 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:05.743915 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:05.743922 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:05.745227 (+ 1305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:05.745588 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:05.745590 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:05.745590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:05.745645 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:05.745876 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:05.745877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:05.745877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:05.745989 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:05.746065 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:05.746222 (+ 157us) server_negotiation.cc:299] Negotiation successful
0812 16:37:05.746302 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 16:37:05.746420 3782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:05.740437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:37355 (local address 127.0.253.193:32921)
0812 16:37:05.740642 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:05.740653 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:05.740708 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:05.740831 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:05.740832 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:05.741047 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:05.741132 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:05.741134 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:05.741974 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:05.741975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:05.742542 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:05.742544 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:05.742919 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:05.742920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:05.742997 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:05.743514 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:05.743520 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:05.743815 (+ 295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:05.745328 (+ 1513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:05.745329 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:05.745330 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:05.745501 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:05.745740 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:05.745741 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:05.745741 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:05.745805 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:05.746057 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:05.746058 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:05.746106 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 16:37:05.746193 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 16:37:05.746304 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":74,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab.
Entry for principal kudu/127.0.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab.
I20260812 16:37:05.766513 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.194:0
--local_ip_for_outbound_sockets=127.0.253.194
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:37355
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:35659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:05.852133 3784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:05.852401 3784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:05.852445 3784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:05.853639 3784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:05.853714 3784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:05.853747 3784 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 16:37:05.853785 3784 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.194
I20260812 16:37:05.855338 3784 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:35659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.253.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.253.254:37355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.194
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:05.855624 3784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:05.855871 3784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:05.861233 3795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:05.861210 3805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:05.861238 3798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:05.861542 3784 server_base.cc:1034] running on GCE node
I20260812 16:37:05.861776 3784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:05.861995 3784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:05.863144 3784 hybrid_clock.cc:648] HybridClock initialized: now 1786552625863114 us; error 43 us; skew 500 ppm
Aug 12 16:37:05 dist-test-slave-w4v5 krb5kdc[3280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552625, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:05.866052 3784 init.cc:377] Logged in from keytab as kudu/127.0.253.194@KRBTEST.COM (short username kudu)
I20260812 16:37:05.866536 3784 webserver.cc:460] Webserver started at http://127.0.253.194:38695/ using document root <none> and password file <none>
I20260812 16:37:05.866798 3784 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:05.866875 3784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:05.867046 3784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:05.868238 3784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-1/data/instance:
uuid: "84c6eda51a614b22bc78dfe6da94141e"
format_stamp: "Formatted at 2026-08-12 16:37:05 on dist-test-slave-w4v5"
server_key: "22e9a5a4e6e251f0ac7c8105138057cb"
server_key_iv: "e37388c0dc3f35eba9ac1e4200481ec3"
server_key_version: "encryptionkey@0"
I20260812 16:37:05.868636 3784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance:
uuid: "84c6eda51a614b22bc78dfe6da94141e"
format_stamp: "Formatted at 2026-08-12 16:37:05 on dist-test-slave-w4v5"
server_key: "22e9a5a4e6e251f0ac7c8105138057cb"
server_key_iv: "e37388c0dc3f35eba9ac1e4200481ec3"
server_key_version: "encryptionkey@0"
I20260812 16:37:05.870111 3784 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:37:05.870940 3817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:05.871191 3784 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:37:05.871310 3784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-1/wal
uuid: "84c6eda51a614b22bc78dfe6da94141e"
format_stamp: "Formatted at 2026-08-12 16:37:05 on dist-test-slave-w4v5"
server_key: "22e9a5a4e6e251f0ac7c8105138057cb"
server_key_iv: "e37388c0dc3f35eba9ac1e4200481ec3"
server_key_version: "encryptionkey@0"
I20260812 16:37:05.871403 3784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:05.896230 3784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:05.899123 3784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:05.899327 3784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:05.899981 3784 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:05.900058 3784 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:05.900110 3784 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:05.900141 3784 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 16:37:05 dist-test-slave-w4v5 krb5kdc[3280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552625, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:05.908418 3784 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.194:42549
I20260812 16:37:05.908771 4029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.194:42549 every 8 connection(s)
I20260812 16:37:05.908905 3784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
I20260812 16:37:05.911504 3461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:05.900640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:36089 (local address 127.0.253.254:37355)
0812 16:37:05.900719 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:05.900722 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:05.900958 (+ 236us) server_negotiation.cc:407] Connection header received
0812 16:37:05.901457 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:05.901459 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:05.901475 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:05.901562 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:05.902617 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:05.903634 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:05.904757 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:05.904977 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:05.905061 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:05.908462 (+ 3401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:05.908465 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:05.908465 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:05.908474 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:05.910023 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:05.910458 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:05.910459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:05.910459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:05.910535 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:05.910802 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:05.910803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:05.910803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:05.910926 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:05.910997 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:05.911328 (+ 331us) server_negotiation.cc:299] Negotiation successful
0812 16:37:05.911411 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 16:37:05.911710 3838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:05.900619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:37355 (local address 127.0.253.194:36089)
0812 16:37:05.900815 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:05.900858 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:05.901353 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:05.901766 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:05.901770 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:05.902188 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:05.902496 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:05.902503 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:05.903751 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:05.903752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:05.904655 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:05.904658 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:05.905081 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:05.905083 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:05.905212 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:05.905824 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:05.905832 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:05.908354 (+ 2522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:05.910122 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:05.910129 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:05.910132 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:05.910362 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:05.910601 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:05.910602 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:05.910603 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:05.910723 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:05.910999 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:05.911007 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:05.911188 (+ 181us) client_negotiation.cc:769] Sending connection context
0812 16:37:05.911305 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 16:37:05.911468 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":119,"threads_started":1}
I20260812 16:37:05.912271 4030 heartbeater.cc:347] Connected to a master server at 127.0.253.254:37355
I20260812 16:37:05.912367 4030 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:05.912568 4030 heartbeater.cc:511] Master 127.0.253.254:37355 requested a full tablet report, sending...
I20260812 16:37:05.912959 3380 ts_manager.cc:194] Registered new tserver with Master: 84c6eda51a614b22bc78dfe6da94141e (127.0.253.194:42549)
I20260812 16:37:05.913122 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 3784
I20260812 16:37:05.913224 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance
I20260812 16:37:05.913403 3380 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.194@KRBTEST.COM'} at 127.0.253.194:36089
I20260812 16:37:05.913439 1015 external_mini_cluster.cc:1428] Setting key 08c38f8eccc87bda8656ab2f39aa7de1
I20260812 16:37:05.920006 3461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:05.913621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:45061 (local address 127.0.253.254:37355)
0812 16:37:05.913665 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:05.913668 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:05.913677 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:37:05.913779 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:05.913780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:05.913789 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:05.913841 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:05.914250 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:05.914921 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:05.915702 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:05.916015 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:05.916087 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:05.917230 (+ 1143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:05.917232 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:05.917232 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:05.917239 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:05.918682 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:05.919143 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:05.919144 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:05.919144 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:05.919237 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:05.919472 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:05.919473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:05.919473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:05.919622 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:05.919680 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:05.919872 (+ 192us) server_negotiation.cc:299] Negotiation successful
0812 16:37:05.919940 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 16:37:05.920102 4040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:05.913376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:37355 (local address 127.0.253.194:45061)
0812 16:37:05.913538 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:05.913552 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:05.913712 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:05.913852 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:05.913854 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:05.914076 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:05.914156 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:05.914159 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:05.915036 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:05.915037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:05.915620 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:05.915621 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:05.916110 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:05.916111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:05.916216 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:05.916631 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:05.916637 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:05.917117 (+ 480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:05.918774 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:05.918775 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:05.918775 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:05.919034 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:05.919320 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:05.919321 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:05.919322 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:05.919401 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:05.919694 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:05.919695 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:05.919757 (+ 62us) client_negotiation.cc:769] Sending connection context
0812 16:37:05.919838 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 16:37:05.919961 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":81,"threads_started":1}
WARNING: no policy specified for kudu/127.0.253.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab.
Entry for principal kudu/127.0.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab.
I20260812 16:37:05.942955 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.195:0
--local_ip_for_outbound_sockets=127.0.253.195
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:37355
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:35659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:06.027560 4046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:06.027776 4046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:06.027799 4046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:06.029002 4046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:06.029049 4046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:06.029069 4046 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 16:37:06.029088 4046 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.195
I20260812 16:37:06.030791 4046 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:35659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.253.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.253.254:37355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.195
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:06.031068 4046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:06.031324 4046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:06.036525 4057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:06.036514 4061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:06.036629 4058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:06.036922 4046 server_base.cc:1034] running on GCE node
I20260812 16:37:06.037178 4046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:06.037438 4046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:06.038615 4046 hybrid_clock.cc:648] HybridClock initialized: now 1786552626038590 us; error 40 us; skew 500 ppm
Aug 12 16:37:06 dist-test-slave-w4v5 krb5kdc[3280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552626, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:06.041517 4046 init.cc:377] Logged in from keytab as kudu/127.0.253.195@KRBTEST.COM (short username kudu)
I20260812 16:37:06.042068 4046 webserver.cc:460] Webserver started at http://127.0.253.195:40095/ using document root <none> and password file <none>
I20260812 16:37:06.042385 4046 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:06.042460 4046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:06.042618 4046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:06.043916 4046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-2/data/instance:
uuid: "db07bd4898fb4d61854ec9f949702c2d"
format_stamp: "Formatted at 2026-08-12 16:37:06 on dist-test-slave-w4v5"
server_key: "8ed8447440bc042073f69f7c04f78a95"
server_key_iv: "8d907a4579db43c177894bc9e0988d3e"
server_key_version: "encryptionkey@0"
I20260812 16:37:06.044308 4046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance:
uuid: "db07bd4898fb4d61854ec9f949702c2d"
format_stamp: "Formatted at 2026-08-12 16:37:06 on dist-test-slave-w4v5"
server_key: "8ed8447440bc042073f69f7c04f78a95"
server_key_iv: "8d907a4579db43c177894bc9e0988d3e"
server_key_version: "encryptionkey@0"
I20260812 16:37:06.045718 4046 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:37:06.046716 4072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:06.046952 4046 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:06.047132 4046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-2/wal
uuid: "db07bd4898fb4d61854ec9f949702c2d"
format_stamp: "Formatted at 2026-08-12 16:37:06 on dist-test-slave-w4v5"
server_key: "8ed8447440bc042073f69f7c04f78a95"
server_key_iv: "8d907a4579db43c177894bc9e0988d3e"
server_key_version: "encryptionkey@0"
I20260812 16:37:06.047230 4046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:06.059723 4046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:06.061656 4046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:06.061821 4046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:06.062392 4046 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:06.062467 4046 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:06.062515 4046 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:06.062546 4046 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 16:37:06 dist-test-slave-w4v5 krb5kdc[3280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552626, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:06.070186 4046 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.195:32825
I20260812 16:37:06.070534 4289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.195:32825 every 8 connection(s)
I20260812 16:37:06.070585 4046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
I20260812 16:37:06.072834 3461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:06.062959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:51725 (local address 127.0.253.254:37355)
0812 16:37:06.063004 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:06.063007 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:06.063294 (+ 287us) server_negotiation.cc:407] Connection header received
0812 16:37:06.063768 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:06.063770 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:06.063786 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:06.063859 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:06.064585 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.065085 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.066118 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.066322 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.066403 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:06.069858 (+ 3455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:06.069860 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:06.069860 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:06.069870 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:06.071625 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:06.071995 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:06.071996 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:06.071997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:06.072050 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:06.072250 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:06.072250 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:06.072250 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:06.072356 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:06.072412 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:06.072700 (+ 288us) server_negotiation.cc:299] Negotiation successful
0812 16:37:06.072770 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 16:37:06.072978 4089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:06.062941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:37355 (local address 127.0.253.195:51725)
0812 16:37:06.063143 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:06.063195 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:06.063667 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:06.063967 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:06.063970 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:06.064273 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:06.064461 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:06.064466 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.065193 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.065193 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:06.066016 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:06.066019 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.066410 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.066411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:06.066649 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:06.067309 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:06.067320 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:06.069757 (+ 2437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:06.071703 (+ 1946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:06.071713 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:06.071715 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:06.071928 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:06.072090 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:06.072091 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:06.072091 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:06.072173 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:06.072420 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:06.072425 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:06.072541 (+ 116us) client_negotiation.cc:769] Sending connection context
0812 16:37:06.072628 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 16:37:06.072785 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":93,"threads_started":1}
I20260812 16:37:06.073475 4290 heartbeater.cc:347] Connected to a master server at 127.0.253.254:37355
I20260812 16:37:06.073563 4290 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:06.073757 4290 heartbeater.cc:511] Master 127.0.253.254:37355 requested a full tablet report, sending...
I20260812 16:37:06.074136 3378 ts_manager.cc:194] Registered new tserver with Master: db07bd4898fb4d61854ec9f949702c2d (127.0.253.195:32825)
I20260812 16:37:06.074621 3378 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.195@KRBTEST.COM'} at 127.0.253.195:51725
I20260812 16:37:06.078570 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 4046
I20260812 16:37:06.078675 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance
I20260812 16:37:06.078842 1015 external_mini_cluster.cc:1428] Setting key a4f26e5e6a962e0a59dcb5562edda0bf
I20260812 16:37:06.079808 1015 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:37:06.080423 4299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:06.074571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:37355 (local address 127.0.253.195:49957)
0812 16:37:06.074702 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:06.074720 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:06.074775 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:06.074906 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:06.074907 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:06.075159 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:06.075234 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:06.075237 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.075954 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.075955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:06.076458 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:06.076460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.076826 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.076827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:06.076900 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:06.077334 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:06.077338 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:06.077637 (+ 299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:06.079345 (+ 1708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:06.079346 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:06.079347 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:06.079575 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:06.079806 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:06.079807 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:06.079807 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:06.079879 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:06.080159 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:06.080160 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:06.080208 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 16:37:06.080262 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 16:37:06.080349 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":72,"threads_started":1}
I20260812 16:37:06.080495 3461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:06.074637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:49957 (local address 127.0.253.254:37355)
0812 16:37:06.074803 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:06.074806 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:06.074814 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:37:06.074833 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:06.074834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:06.074843 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:06.074902 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:06.075300 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.075867 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.076530 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.076742 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.076795 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:06.077738 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:06.077740 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:06.077740 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:06.077746 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:06.079259 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:06.079661 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:06.079663 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:06.079663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:06.079723 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:06.079953 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:06.079954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:06.079954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:06.080090 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:06.080162 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:06.080318 (+ 156us) server_negotiation.cc:299] Negotiation successful
0812 16:37:06.080408 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148}
Aug 12 16:37:06 dist-test-slave-w4v5 krb5kdc[3280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552626, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 16:37:06 dist-test-slave-w4v5 krb5kdc[3280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552626, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:06.104068 3461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:06.097191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55608 (local address 127.0.253.254:37355)
0812 16:37:06.097263 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:06.097265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:06.097274 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:37:06.097299 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:06.097300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:06.097310 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:06.097360 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:06.097939 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.098446 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.099308 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.099496 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.099556 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:06.101758 (+ 2202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:06.101760 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:06.101760 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:06.101768 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:06.103019 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:06.103427 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:06.103428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:06.103428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:06.103484 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:06.103653 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:06.103654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:06.103654 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:06.103763 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:06.103845 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:06.103935 (+ 90us) server_negotiation.cc:299] Negotiation successful
0812 16:37:06.103997 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 16:37:06.105868 3378 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55608:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:37:06.107795 3378 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:37:06.114965 4335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:06.111000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55480 (local address 127.0.253.194:42549)
0812 16:37:06.111239 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:06.111243 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:06.111253 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:37:06.111382 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:06.111384 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:06.111403 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:06.111480 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:06.111679 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.112779 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.114025 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.114522 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.114614 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:06.114693 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:06.114811 (+ 118us) server_negotiation.cc:299] Negotiation successful
0812 16:37:06.114850 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":113,"threads_started":1}
I20260812 16:37:06.114949 4327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:06.110444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.195:32825 (local address 127.0.0.1:46592)
0812 16:37:06.110667 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:06.110713 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:06.110792 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:06.111160 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:06.111164 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:06.111167 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:06.111245 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:06.111247 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.112656 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.112657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:06.113693 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:06.113695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.114658 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.114659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:06.114722 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:06.114727 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:37:06.114764 (+ 37us) client_negotiation.cc:240] Negotiation successful
0812 16:37:06.114831 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":2,"thread_start_us":93,"threads_started":1}
I20260812 16:37:06.115166 4332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:06.110579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33718 (local address 127.0.253.193:38327)
0812 16:37:06.110773 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:06.110779 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:06.111023 (+ 244us) server_negotiation.cc:407] Connection header received
0812 16:37:06.111109 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:06.111113 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:06.111132 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:06.111224 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:06.111450 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.112322 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.113844 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.114569 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.114633 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:06.114693 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:06.114988 (+ 295us) server_negotiation.cc:299] Negotiation successful
0812 16:37:06.115030 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":86,"threads_started":1}
I20260812 16:37:06.114949 4323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:06.110736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:42549 (local address 127.0.0.1:55480)
0812 16:37:06.110899 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:06.110908 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:06.110946 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:06.111548 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:06.111549 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:06.111551 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:06.111586 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:06.111587 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.112869 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.112870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:06.113937 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:06.113938 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.114613 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.114614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:06.114701 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:06.114717 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 16:37:06.114795 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 16:37:06.114841 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":97,"threads_started":1}
I20260812 16:37:06.115594 4331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:06.110558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46592 (local address 127.0.253.195:32825)
0812 16:37:06.110865 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:06.110870 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:06.110882 (+ 12us) server_negotiation.cc:407] Connection header received
0812 16:37:06.110930 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:06.110932 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:06.110949 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:06.111033 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:06.111339 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.112544 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.113777 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.114552 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.115003 (+ 451us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:06.115404 (+ 401us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:06.115427 (+ 23us) server_negotiation.cc:299] Negotiation successful
0812 16:37:06.115496 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":100,"threads_started":1}
I20260812 16:37:06.114949 4334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:06.110455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:38327 (local address 127.0.0.1:33718)
0812 16:37:06.110874 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:06.110887 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:06.110952 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:06.111307 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:06.111308 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:06.111310 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:06.111362 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:06.111363 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.112433 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.112435 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:06.113749 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:06.113752 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.114722 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.114723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:06.114778 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:06.114780 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 16:37:06.114826 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 16:37:06.114859 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"mutex_wait_us":40,"thread_start_us":129,"threads_started":1}
I20260812 16:37:06.116205 4159 tablet_service.cc:1467] Processing CreateTablet for tablet b8cb9d8e7654408fbf1cfff15e108cce (DEFAULT_TABLE table=test-table [id=7e5729b0695a4f84a2c142757518bf4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:06.116145 3904 tablet_service.cc:1467] Processing CreateTablet for tablet b8cb9d8e7654408fbf1cfff15e108cce (DEFAULT_TABLE table=test-table [id=7e5729b0695a4f84a2c142757518bf4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:06.116583 3904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8cb9d8e7654408fbf1cfff15e108cce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:06.116578 4159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8cb9d8e7654408fbf1cfff15e108cce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:06.120110 4337 tablet_bootstrap.cc:492] T b8cb9d8e7654408fbf1cfff15e108cce P db07bd4898fb4d61854ec9f949702c2d: Bootstrap starting.
I20260812 16:37:06.120375 4341 tablet_bootstrap.cc:492] T b8cb9d8e7654408fbf1cfff15e108cce P 84c6eda51a614b22bc78dfe6da94141e: Bootstrap starting.
I20260812 16:37:06.121037 3645 tablet_service.cc:1467] Processing CreateTablet for tablet b8cb9d8e7654408fbf1cfff15e108cce (DEFAULT_TABLE table=test-table [id=7e5729b0695a4f84a2c142757518bf4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:06.121151 4341 tablet_bootstrap.cc:654] T b8cb9d8e7654408fbf1cfff15e108cce P 84c6eda51a614b22bc78dfe6da94141e: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:06.121151 4337 tablet_bootstrap.cc:654] T b8cb9d8e7654408fbf1cfff15e108cce P db07bd4898fb4d61854ec9f949702c2d: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:06.121413 3645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8cb9d8e7654408fbf1cfff15e108cce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:06.121528 4341 log.cc:824] T b8cb9d8e7654408fbf1cfff15e108cce P 84c6eda51a614b22bc78dfe6da94141e: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:06.121528 4337 log.cc:824] T b8cb9d8e7654408fbf1cfff15e108cce P db07bd4898fb4d61854ec9f949702c2d: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:06.122390 4337 tablet_bootstrap.cc:492] T b8cb9d8e7654408fbf1cfff15e108cce P db07bd4898fb4d61854ec9f949702c2d: No bootstrap required, opened a new log
I20260812 16:37:06.122476 4337 ts_tablet_manager.cc:1397] T b8cb9d8e7654408fbf1cfff15e108cce P db07bd4898fb4d61854ec9f949702c2d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:37:06.122591 4341 tablet_bootstrap.cc:492] T b8cb9d8e7654408fbf1cfff15e108cce P 84c6eda51a614b22bc78dfe6da94141e: No bootstrap required, opened a new log
I20260812 16:37:06.122668 4341 ts_tablet_manager.cc:1397] T b8cb9d8e7654408fbf1cfff15e108cce P 84c6eda51a614b22bc78dfe6da94141e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:37:06.123988 4337 raft_consensus.cc:348] T b8cb9d8e7654408fbf1cfff15e108cce P db07bd4898fb4d61854ec9f949702c2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db07bd4898fb4d61854ec9f949702c2d" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32825 } } peers { permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 } } peers { permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 } }
I20260812 16:37:06.124166 4337 raft_consensus.cc:374] T b8cb9d8e7654408fbf1cfff15e108cce P db07bd4898fb4d61854ec9f949702c2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:06.124213 4337 raft_consensus.cc:729] T b8cb9d8e7654408fbf1cfff15e108cce P db07bd4898fb4d61854ec9f949702c2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db07bd4898fb4d61854ec9f949702c2d, State: Initialized, Role: FOLLOWER
I20260812 16:37:06.124337 4337 consensus_queue.cc:260] T b8cb9d8e7654408fbf1cfff15e108cce P db07bd4898fb4d61854ec9f949702c2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db07bd4898fb4d61854ec9f949702c2d" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32825 } } peers { permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 } } peers { permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 } }
I20260812 16:37:06.124619 4337 ts_tablet_manager.cc:1428] T b8cb9d8e7654408fbf1cfff15e108cce P db07bd4898fb4d61854ec9f949702c2d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:37:06.124763 4290 heartbeater.cc:503] Master 127.0.253.254:37355 was elected leader, sending a full tablet report...
I20260812 16:37:06.124789 4341 raft_consensus.cc:348] T b8cb9d8e7654408fbf1cfff15e108cce P 84c6eda51a614b22bc78dfe6da94141e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db07bd4898fb4d61854ec9f949702c2d" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32825 } } peers { permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 } } peers { permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 } }
I20260812 16:37:06.124954 4341 raft_consensus.cc:374] T b8cb9d8e7654408fbf1cfff15e108cce P 84c6eda51a614b22bc78dfe6da94141e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:06.125007 4341 raft_consensus.cc:729] T b8cb9d8e7654408fbf1cfff15e108cce P 84c6eda51a614b22bc78dfe6da94141e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84c6eda51a614b22bc78dfe6da94141e, State: Initialized, Role: FOLLOWER
I20260812 16:37:06.125133 4341 consensus_queue.cc:260] T b8cb9d8e7654408fbf1cfff15e108cce P 84c6eda51a614b22bc78dfe6da94141e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db07bd4898fb4d61854ec9f949702c2d" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32825 } } peers { permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 } } peers { permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 } }
I20260812 16:37:06.125401 4341 ts_tablet_manager.cc:1428] T b8cb9d8e7654408fbf1cfff15e108cce P 84c6eda51a614b22bc78dfe6da94141e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:37:06.125444 4030 heartbeater.cc:503] Master 127.0.253.254:37355 was elected leader, sending a full tablet report...
I20260812 16:37:06.125730 4354 tablet_bootstrap.cc:492] T b8cb9d8e7654408fbf1cfff15e108cce P d5d3d26c2a9b4f8c8f839739e0074511: Bootstrap starting.
I20260812 16:37:06.126421 4354 tablet_bootstrap.cc:654] T b8cb9d8e7654408fbf1cfff15e108cce P d5d3d26c2a9b4f8c8f839739e0074511: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:06.126714 4354 log.cc:824] T b8cb9d8e7654408fbf1cfff15e108cce P d5d3d26c2a9b4f8c8f839739e0074511: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:06.127470 4354 tablet_bootstrap.cc:492] T b8cb9d8e7654408fbf1cfff15e108cce P d5d3d26c2a9b4f8c8f839739e0074511: No bootstrap required, opened a new log
I20260812 16:37:06.127552 4354 ts_tablet_manager.cc:1397] T b8cb9d8e7654408fbf1cfff15e108cce P d5d3d26c2a9b4f8c8f839739e0074511: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:37:06.128868 4354 raft_consensus.cc:348] T b8cb9d8e7654408fbf1cfff15e108cce P d5d3d26c2a9b4f8c8f839739e0074511 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db07bd4898fb4d61854ec9f949702c2d" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32825 } } peers { permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 } } peers { permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 } }
I20260812 16:37:06.129011 4354 raft_consensus.cc:374] T b8cb9d8e7654408fbf1cfff15e108cce P d5d3d26c2a9b4f8c8f839739e0074511 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:06.129067 4354 raft_consensus.cc:729] T b8cb9d8e7654408fbf1cfff15e108cce P d5d3d26c2a9b4f8c8f839739e0074511 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5d3d26c2a9b4f8c8f839739e0074511, State: Initialized, Role: FOLLOWER
I20260812 16:37:06.129205 4354 consensus_queue.cc:260] T b8cb9d8e7654408fbf1cfff15e108cce P d5d3d26c2a9b4f8c8f839739e0074511 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db07bd4898fb4d61854ec9f949702c2d" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32825 } } peers { permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 } } peers { permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 } }
I20260812 16:37:06.129446 4354 ts_tablet_manager.cc:1428] T b8cb9d8e7654408fbf1cfff15e108cce P d5d3d26c2a9b4f8c8f839739e0074511: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:37:06.129559 3773 heartbeater.cc:503] Master 127.0.253.254:37355 was elected leader, sending a full tablet report...
I20260812 16:37:06.142802 4349 raft_consensus.cc:482] T b8cb9d8e7654408fbf1cfff15e108cce P db07bd4898fb4d61854ec9f949702c2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:37:06.142942 4349 raft_consensus.cc:504] T b8cb9d8e7654408fbf1cfff15e108cce P db07bd4898fb4d61854ec9f949702c2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db07bd4898fb4d61854ec9f949702c2d" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32825 } } peers { permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 } } peers { permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 } }
I20260812 16:37:06.143350 4349 leader_election.cc:288] T b8cb9d8e7654408fbf1cfff15e108cce P db07bd4898fb4d61854ec9f949702c2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84c6eda51a614b22bc78dfe6da94141e (127.0.253.194:42549), d5d3d26c2a9b4f8c8f839739e0074511 (127.0.253.193:38327)
I20260812 16:37:06.146618 4089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:06.143573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:38327 (local address 127.0.253.195:54915)
0812 16:37:06.143686 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:06.143704 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:06.143766 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:06.143934 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:06.143935 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:06.143937 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:06.144025 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:06.144027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.144746 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.144747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:06.145594 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:06.145597 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.146331 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.146332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:06.146439 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:06.146444 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:37:06.146505 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 16:37:06.146549 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"mutex_wait_us":1}
I20260812 16:37:06.146752 4332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:06.143726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:54915 (local address 127.0.253.193:38327)
0812 16:37:06.143792 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:06.143794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:06.143808 (+ 14us) server_negotiation.cc:407] Connection header received
0812 16:37:06.143839 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:06.143840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:06.143850 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:06.143911 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:06.144131 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.144636 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.145690 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.146253 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.146310 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:06.146318 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:06.146635 (+ 317us) server_negotiation.cc:299] Negotiation successful
0812 16:37:06.146678 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 16:37:06.146973 4335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:06.143764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:51991 (local address 127.0.253.194:42549)
0812 16:37:06.143815 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:06.143818 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:06.143858 (+ 40us) server_negotiation.cc:407] Connection header received
0812 16:37:06.143944 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:06.143946 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:06.143956 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:06.144025 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:06.144206 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.144739 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.145750 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.146412 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.146497 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:06.146507 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:06.146845 (+ 338us) server_negotiation.cc:299] Negotiation successful
0812 16:37:06.146895 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 16:37:06.147114 3681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8cb9d8e7654408fbf1cfff15e108cce" candidate_uuid: "db07bd4898fb4d61854ec9f949702c2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" is_pre_election: true
I20260812 16:37:06.147275 4357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:06.143572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:42549 (local address 127.0.253.195:51991)
0812 16:37:06.143795 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:06.143806 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:06.143873 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:06.144034 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:06.144035 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:06.144036 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:06.144127 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:06.144129 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.144824 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.144825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:06.145622 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:06.145625 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.146656 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.146657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:06.146738 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:06.146742 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:37:06.147142 (+ 400us) client_negotiation.cc:240] Negotiation successful
0812 16:37:06.147187 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":37,"thread_start_us":122,"threads_started":1}
I20260812 16:37:06.147290 3681 raft_consensus.cc:2454] T b8cb9d8e7654408fbf1cfff15e108cce P d5d3d26c2a9b4f8c8f839739e0074511 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db07bd4898fb4d61854ec9f949702c2d in term 0.
I20260812 16:37:06.147548 4074 leader_election.cc:302] T b8cb9d8e7654408fbf1cfff15e108cce P db07bd4898fb4d61854ec9f949702c2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d5d3d26c2a9b4f8c8f839739e0074511, db07bd4898fb4d61854ec9f949702c2d; no voters:
I20260812 16:37:06.147701 4349 raft_consensus.cc:2781] T b8cb9d8e7654408fbf1cfff15e108cce P db07bd4898fb4d61854ec9f949702c2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:37:06.147673 3939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8cb9d8e7654408fbf1cfff15e108cce" candidate_uuid: "db07bd4898fb4d61854ec9f949702c2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84c6eda51a614b22bc78dfe6da94141e" is_pre_election: true
I20260812 16:37:06.147768 4349 raft_consensus.cc:482] T b8cb9d8e7654408fbf1cfff15e108cce P db07bd4898fb4d61854ec9f949702c2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:37:06.147830 4349 raft_consensus.cc:3037] T b8cb9d8e7654408fbf1cfff15e108cce P db07bd4898fb4d61854ec9f949702c2d [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:06.147825 3939 raft_consensus.cc:2454] T b8cb9d8e7654408fbf1cfff15e108cce P 84c6eda51a614b22bc78dfe6da94141e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db07bd4898fb4d61854ec9f949702c2d in term 0.
I20260812 16:37:06.148869 4349 raft_consensus.cc:504] T b8cb9d8e7654408fbf1cfff15e108cce P db07bd4898fb4d61854ec9f949702c2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db07bd4898fb4d61854ec9f949702c2d" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32825 } } peers { permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 } } peers { permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 } }
I20260812 16:37:06.149044 4349 leader_election.cc:288] T b8cb9d8e7654408fbf1cfff15e108cce P db07bd4898fb4d61854ec9f949702c2d [CANDIDATE]: Term 1 election: Requested vote from peers 84c6eda51a614b22bc78dfe6da94141e (127.0.253.194:42549), d5d3d26c2a9b4f8c8f839739e0074511 (127.0.253.193:38327)
I20260812 16:37:06.149204 3939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8cb9d8e7654408fbf1cfff15e108cce" candidate_uuid: "db07bd4898fb4d61854ec9f949702c2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84c6eda51a614b22bc78dfe6da94141e"
I20260812 16:37:06.149235 3681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8cb9d8e7654408fbf1cfff15e108cce" candidate_uuid: "db07bd4898fb4d61854ec9f949702c2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5d3d26c2a9b4f8c8f839739e0074511"
I20260812 16:37:06.149302 3681 raft_consensus.cc:3037] T b8cb9d8e7654408fbf1cfff15e108cce P d5d3d26c2a9b4f8c8f839739e0074511 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:06.149293 3939 raft_consensus.cc:3037] T b8cb9d8e7654408fbf1cfff15e108cce P 84c6eda51a614b22bc78dfe6da94141e [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:06.150276 3939 raft_consensus.cc:2454] T b8cb9d8e7654408fbf1cfff15e108cce P 84c6eda51a614b22bc78dfe6da94141e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db07bd4898fb4d61854ec9f949702c2d in term 1.
I20260812 16:37:06.150275 3681 raft_consensus.cc:2454] T b8cb9d8e7654408fbf1cfff15e108cce P d5d3d26c2a9b4f8c8f839739e0074511 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db07bd4898fb4d61854ec9f949702c2d in term 1.
I20260812 16:37:06.150540 4077 leader_election.cc:302] T b8cb9d8e7654408fbf1cfff15e108cce P db07bd4898fb4d61854ec9f949702c2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84c6eda51a614b22bc78dfe6da94141e, db07bd4898fb4d61854ec9f949702c2d; no voters:
I20260812 16:37:06.150633 4349 raft_consensus.cc:2781] T b8cb9d8e7654408fbf1cfff15e108cce P db07bd4898fb4d61854ec9f949702c2d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:37:06.150857 4349 raft_consensus.cc:686] T b8cb9d8e7654408fbf1cfff15e108cce P db07bd4898fb4d61854ec9f949702c2d [term 1 LEADER]: Becoming Leader. State: Replica: db07bd4898fb4d61854ec9f949702c2d, State: Running, Role: LEADER
I20260812 16:37:06.150983 4349 consensus_queue.cc:237] T b8cb9d8e7654408fbf1cfff15e108cce P db07bd4898fb4d61854ec9f949702c2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db07bd4898fb4d61854ec9f949702c2d" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32825 } } peers { permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 } } peers { permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 } }
I20260812 16:37:06.151997 3378 catalog_manager.cc:5495] T b8cb9d8e7654408fbf1cfff15e108cce P db07bd4898fb4d61854ec9f949702c2d reported cstate change: term changed from 0 to 1, leader changed from <none> to db07bd4898fb4d61854ec9f949702c2d (127.0.253.195). New cstate: current_term: 1 leader_uuid: "db07bd4898fb4d61854ec9f949702c2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db07bd4898fb4d61854ec9f949702c2d" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32825 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 } health_report { overall_health: UNKNOWN } } }
W20260812 16:37:06.159569 4031 tablet.cc:2369] T b8cb9d8e7654408fbf1cfff15e108cce P 84c6eda51a614b22bc78dfe6da94141e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:37:06.162191 4331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:06.159583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46604 (local address 127.0.253.195:32825)
0812 16:37:06.159641 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:06.159643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:06.159673 (+ 30us) server_negotiation.cc:407] Connection header received
0812 16:37:06.159781 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:06.159782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:06.159799 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:06.159854 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:37:06.160072 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.160553 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.161331 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.161533 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.161599 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:06.161818 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:37:06.161929 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:37:06.162003 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:06.162046 (+ 43us) server_negotiation.cc:299] Negotiation successful
0812 16:37:06.162070 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 16:37:06.164402 3939 raft_consensus.cc:1261] T b8cb9d8e7654408fbf1cfff15e108cce P 84c6eda51a614b22bc78dfe6da94141e [term 1 FOLLOWER]: Refusing update from remote peer db07bd4898fb4d61854ec9f949702c2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:37:06.164819 4358 consensus_queue.cc:1035] T b8cb9d8e7654408fbf1cfff15e108cce P db07bd4898fb4d61854ec9f949702c2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:37:06.166922 3681 raft_consensus.cc:1261] T b8cb9d8e7654408fbf1cfff15e108cce P d5d3d26c2a9b4f8c8f839739e0074511 [term 1 FOLLOWER]: Refusing update from remote peer db07bd4898fb4d61854ec9f949702c2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:37:06.167373 4358 consensus_queue.cc:1035] T b8cb9d8e7654408fbf1cfff15e108cce P db07bd4898fb4d61854ec9f949702c2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:37:06.167904 4364 mvcc.cc:205] Tried to move back new op lower bound from 7317719556765831168 to 7317719556715495424. Current Snapshot: MvccSnapshot[applied={T|T < 7317719556765831168}]
I20260812 16:37:06.172155 3378 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55608:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:37:06.172299 3378 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55608:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:37:06.173223 3378 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 80e95251a626458bb717456269c4ea24: Sending DeleteTablet for 3 replicas of tablet b8cb9d8e7654408fbf1cfff15e108cce
I20260812 16:37:06.173755 3645 tablet_service.cc:1514] Processing DeleteTablet for tablet b8cb9d8e7654408fbf1cfff15e108cce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:06 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:33718
I20260812 16:37:06.173748 3904 tablet_service.cc:1514] Processing DeleteTablet for tablet b8cb9d8e7654408fbf1cfff15e108cce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:06 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:55480
I20260812 16:37:06.174016 4380 tablet_replica.cc:330] T b8cb9d8e7654408fbf1cfff15e108cce P d5d3d26c2a9b4f8c8f839739e0074511: stopping tablet replica
I20260812 16:37:06.174116 4380 raft_consensus.cc:2229] T b8cb9d8e7654408fbf1cfff15e108cce P d5d3d26c2a9b4f8c8f839739e0074511 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:37:06.174237 4380 raft_consensus.cc:2258] T b8cb9d8e7654408fbf1cfff15e108cce P d5d3d26c2a9b4f8c8f839739e0074511 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:06.174269 4383 tablet_replica.cc:330] T b8cb9d8e7654408fbf1cfff15e108cce P 84c6eda51a614b22bc78dfe6da94141e: stopping tablet replica
I20260812 16:37:06.174422 4383 raft_consensus.cc:2229] T b8cb9d8e7654408fbf1cfff15e108cce P 84c6eda51a614b22bc78dfe6da94141e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:37:06.174713 4383 raft_consensus.cc:2258] T b8cb9d8e7654408fbf1cfff15e108cce P 84c6eda51a614b22bc78dfe6da94141e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:06.174840 4380 ts_tablet_manager.cc:1905] T b8cb9d8e7654408fbf1cfff15e108cce P d5d3d26c2a9b4f8c8f839739e0074511: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:37:06.175362 4383 ts_tablet_manager.cc:1905] T b8cb9d8e7654408fbf1cfff15e108cce P 84c6eda51a614b22bc78dfe6da94141e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:37:06.175823 4159 tablet_service.cc:1514] Processing DeleteTablet for tablet b8cb9d8e7654408fbf1cfff15e108cce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:06 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:46592
I20260812 16:37:06.176352 3378 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55608:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:37:06.176519 3378 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:37:06.177146 4383 ts_tablet_manager.cc:1918] T b8cb9d8e7654408fbf1cfff15e108cce P 84c6eda51a614b22bc78dfe6da94141e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:37:06.177235 4383 log.cc:1196] T b8cb9d8e7654408fbf1cfff15e108cce P 84c6eda51a614b22bc78dfe6da94141e: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-1/wal/wals/b8cb9d8e7654408fbf1cfff15e108cce
I20260812 16:37:06.177487 4380 ts_tablet_manager.cc:1918] T b8cb9d8e7654408fbf1cfff15e108cce P d5d3d26c2a9b4f8c8f839739e0074511: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:37:06.177539 4383 ts_tablet_manager.cc:1939] T b8cb9d8e7654408fbf1cfff15e108cce P 84c6eda51a614b22bc78dfe6da94141e: Deleting consensus metadata
I20260812 16:37:06.177665 4380 log.cc:1196] T b8cb9d8e7654408fbf1cfff15e108cce P d5d3d26c2a9b4f8c8f839739e0074511: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-0/wal/wals/b8cb9d8e7654408fbf1cfff15e108cce
I20260812 16:37:06.177919 3645 tablet_service.cc:1467] Processing CreateTablet for tablet c187fe17c9124d4d9a35d5ec761ce375 (DEFAULT_TABLE table=test-table [id=164bbe7feb524cf390c3f996ed717c65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:06.178027 3645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c187fe17c9124d4d9a35d5ec761ce375. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:06.178026 3904 tablet_service.cc:1467] Processing CreateTablet for tablet c187fe17c9124d4d9a35d5ec761ce375 (DEFAULT_TABLE table=test-table [id=164bbe7feb524cf390c3f996ed717c65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:06.178133 3904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c187fe17c9124d4d9a35d5ec761ce375. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:06.178268 4386 tablet_replica.cc:330] T b8cb9d8e7654408fbf1cfff15e108cce P db07bd4898fb4d61854ec9f949702c2d: stopping tablet replica
I20260812 16:37:06.178357 4386 raft_consensus.cc:2229] T b8cb9d8e7654408fbf1cfff15e108cce P db07bd4898fb4d61854ec9f949702c2d [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:37:06.178474 4386 raft_consensus.cc:2258] T b8cb9d8e7654408fbf1cfff15e108cce P db07bd4898fb4d61854ec9f949702c2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:06.179116 4354 tablet_bootstrap.cc:492] T c187fe17c9124d4d9a35d5ec761ce375 P d5d3d26c2a9b4f8c8f839739e0074511: Bootstrap starting.
I20260812 16:37:06.179450 4341 tablet_bootstrap.cc:492] T c187fe17c9124d4d9a35d5ec761ce375 P 84c6eda51a614b22bc78dfe6da94141e: Bootstrap starting.
I20260812 16:37:06.179582 4354 tablet_bootstrap.cc:654] T c187fe17c9124d4d9a35d5ec761ce375 P d5d3d26c2a9b4f8c8f839739e0074511: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:06.179603 4159 tablet_service.cc:1467] Processing CreateTablet for tablet c187fe17c9124d4d9a35d5ec761ce375 (DEFAULT_TABLE table=test-table [id=164bbe7feb524cf390c3f996ed717c65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:06.179744 4159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c187fe17c9124d4d9a35d5ec761ce375. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:06.179881 4341 tablet_bootstrap.cc:654] T c187fe17c9124d4d9a35d5ec761ce375 P 84c6eda51a614b22bc78dfe6da94141e: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:06.180536 4380 ts_tablet_manager.cc:1939] T b8cb9d8e7654408fbf1cfff15e108cce P d5d3d26c2a9b4f8c8f839739e0074511: Deleting consensus metadata
I20260812 16:37:06.181010 3355 catalog_manager.cc:4841] TS d5d3d26c2a9b4f8c8f839739e0074511 (127.0.253.193:38327): tablet b8cb9d8e7654408fbf1cfff15e108cce (table test-table [id=7e5729b0695a4f84a2c142757518bf4b]) successfully deleted
I20260812 16:37:06.181414 4386 ts_tablet_manager.cc:1905] T b8cb9d8e7654408fbf1cfff15e108cce P db07bd4898fb4d61854ec9f949702c2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:37:06.181453 4337 tablet_bootstrap.cc:492] T c187fe17c9124d4d9a35d5ec761ce375 P db07bd4898fb4d61854ec9f949702c2d: Bootstrap starting.
I20260812 16:37:06.181825 3359 catalog_manager.cc:4841] TS 84c6eda51a614b22bc78dfe6da94141e (127.0.253.194:42549): tablet b8cb9d8e7654408fbf1cfff15e108cce (table test-table [id=7e5729b0695a4f84a2c142757518bf4b]) successfully deleted
I20260812 16:37:06.182010 4337 tablet_bootstrap.cc:654] T c187fe17c9124d4d9a35d5ec761ce375 P db07bd4898fb4d61854ec9f949702c2d: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:06.182765 4337 tablet_bootstrap.cc:492] T c187fe17c9124d4d9a35d5ec761ce375 P db07bd4898fb4d61854ec9f949702c2d: No bootstrap required, opened a new log
I20260812 16:37:06.182825 4337 ts_tablet_manager.cc:1397] T c187fe17c9124d4d9a35d5ec761ce375 P db07bd4898fb4d61854ec9f949702c2d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:06.183000 4337 raft_consensus.cc:348] T c187fe17c9124d4d9a35d5ec761ce375 P db07bd4898fb4d61854ec9f949702c2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 } } peers { permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 } } peers { permanent_uuid: "db07bd4898fb4d61854ec9f949702c2d" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32825 } }
I20260812 16:37:06.183125 4337 raft_consensus.cc:374] T c187fe17c9124d4d9a35d5ec761ce375 P db07bd4898fb4d61854ec9f949702c2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:06.183152 4337 raft_consensus.cc:729] T c187fe17c9124d4d9a35d5ec761ce375 P db07bd4898fb4d61854ec9f949702c2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db07bd4898fb4d61854ec9f949702c2d, State: Initialized, Role: FOLLOWER
I20260812 16:37:06.183200 4337 consensus_queue.cc:260] T c187fe17c9124d4d9a35d5ec761ce375 P db07bd4898fb4d61854ec9f949702c2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 } } peers { permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 } } peers { permanent_uuid: "db07bd4898fb4d61854ec9f949702c2d" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32825 } }
I20260812 16:37:06.183287 4341 tablet_bootstrap.cc:492] T c187fe17c9124d4d9a35d5ec761ce375 P 84c6eda51a614b22bc78dfe6da94141e: No bootstrap required, opened a new log
I20260812 16:37:06.183331 4341 ts_tablet_manager.cc:1397] T c187fe17c9124d4d9a35d5ec761ce375 P 84c6eda51a614b22bc78dfe6da94141e: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260812 16:37:06.183350 4337 ts_tablet_manager.cc:1428] T c187fe17c9124d4d9a35d5ec761ce375 P db07bd4898fb4d61854ec9f949702c2d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 16:37:06.183178 4386 ts_tablet_manager.cc:1918] T b8cb9d8e7654408fbf1cfff15e108cce P db07bd4898fb4d61854ec9f949702c2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:37:06.183458 4386 log.cc:1196] T b8cb9d8e7654408fbf1cfff15e108cce P db07bd4898fb4d61854ec9f949702c2d: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-2/wal/wals/b8cb9d8e7654408fbf1cfff15e108cce
I20260812 16:37:06.183526 4341 raft_consensus.cc:348] T c187fe17c9124d4d9a35d5ec761ce375 P 84c6eda51a614b22bc78dfe6da94141e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 } } peers { permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 } } peers { permanent_uuid: "db07bd4898fb4d61854ec9f949702c2d" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32825 } }
I20260812 16:37:06.183610 4341 raft_consensus.cc:374] T c187fe17c9124d4d9a35d5ec761ce375 P 84c6eda51a614b22bc78dfe6da94141e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:06.183636 4341 raft_consensus.cc:729] T c187fe17c9124d4d9a35d5ec761ce375 P 84c6eda51a614b22bc78dfe6da94141e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84c6eda51a614b22bc78dfe6da94141e, State: Initialized, Role: FOLLOWER
I20260812 16:37:06.183686 4341 consensus_queue.cc:260] T c187fe17c9124d4d9a35d5ec761ce375 P 84c6eda51a614b22bc78dfe6da94141e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 } } peers { permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 } } peers { permanent_uuid: "db07bd4898fb4d61854ec9f949702c2d" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32825 } }
I20260812 16:37:06.183730 4386 ts_tablet_manager.cc:1939] T b8cb9d8e7654408fbf1cfff15e108cce P db07bd4898fb4d61854ec9f949702c2d: Deleting consensus metadata
I20260812 16:37:06.183801 4341 ts_tablet_manager.cc:1428] T c187fe17c9124d4d9a35d5ec761ce375 P 84c6eda51a614b22bc78dfe6da94141e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 16:37:06.184451 3359 catalog_manager.cc:4841] TS db07bd4898fb4d61854ec9f949702c2d (127.0.253.195:32825): tablet b8cb9d8e7654408fbf1cfff15e108cce (table test-table [id=7e5729b0695a4f84a2c142757518bf4b]) successfully deleted
I20260812 16:37:06.184752 4354 tablet_bootstrap.cc:492] T c187fe17c9124d4d9a35d5ec761ce375 P d5d3d26c2a9b4f8c8f839739e0074511: No bootstrap required, opened a new log
I20260812 16:37:06.184808 4354 ts_tablet_manager.cc:1397] T c187fe17c9124d4d9a35d5ec761ce375 P d5d3d26c2a9b4f8c8f839739e0074511: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260812 16:37:06.184980 4354 raft_consensus.cc:348] T c187fe17c9124d4d9a35d5ec761ce375 P d5d3d26c2a9b4f8c8f839739e0074511 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 } } peers { permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 } } peers { permanent_uuid: "db07bd4898fb4d61854ec9f949702c2d" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32825 } }
I20260812 16:37:06.185078 4354 raft_consensus.cc:374] T c187fe17c9124d4d9a35d5ec761ce375 P d5d3d26c2a9b4f8c8f839739e0074511 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:06.185103 4354 raft_consensus.cc:729] T c187fe17c9124d4d9a35d5ec761ce375 P d5d3d26c2a9b4f8c8f839739e0074511 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5d3d26c2a9b4f8c8f839739e0074511, State: Initialized, Role: FOLLOWER
I20260812 16:37:06.185155 4354 consensus_queue.cc:260] T c187fe17c9124d4d9a35d5ec761ce375 P d5d3d26c2a9b4f8c8f839739e0074511 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 } } peers { permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 } } peers { permanent_uuid: "db07bd4898fb4d61854ec9f949702c2d" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32825 } }
I20260812 16:37:06.185246 4354 ts_tablet_manager.cc:1428] T c187fe17c9124d4d9a35d5ec761ce375 P d5d3d26c2a9b4f8c8f839739e0074511: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
W20260812 16:37:06.235046 3774 tablet.cc:2369] T c187fe17c9124d4d9a35d5ec761ce375 P d5d3d26c2a9b4f8c8f839739e0074511: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 16:37:06.321216 4291 tablet.cc:2369] T c187fe17c9124d4d9a35d5ec761ce375 P db07bd4898fb4d61854ec9f949702c2d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:37:06.401067 4349 raft_consensus.cc:482] T c187fe17c9124d4d9a35d5ec761ce375 P db07bd4898fb4d61854ec9f949702c2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:37:06.401181 4349 raft_consensus.cc:504] T c187fe17c9124d4d9a35d5ec761ce375 P db07bd4898fb4d61854ec9f949702c2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 } } peers { permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 } } peers { permanent_uuid: "db07bd4898fb4d61854ec9f949702c2d" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32825 } }
I20260812 16:37:06.401384 4349 leader_election.cc:288] T c187fe17c9124d4d9a35d5ec761ce375 P db07bd4898fb4d61854ec9f949702c2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5d3d26c2a9b4f8c8f839739e0074511 (127.0.253.193:38327), 84c6eda51a614b22bc78dfe6da94141e (127.0.253.194:42549)
I20260812 16:37:06.401697 3681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c187fe17c9124d4d9a35d5ec761ce375" candidate_uuid: "db07bd4898fb4d61854ec9f949702c2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" is_pre_election: true
I20260812 16:37:06.401742 3939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c187fe17c9124d4d9a35d5ec761ce375" candidate_uuid: "db07bd4898fb4d61854ec9f949702c2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84c6eda51a614b22bc78dfe6da94141e" is_pre_election: true
I20260812 16:37:06.401835 3939 raft_consensus.cc:2454] T c187fe17c9124d4d9a35d5ec761ce375 P 84c6eda51a614b22bc78dfe6da94141e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db07bd4898fb4d61854ec9f949702c2d in term 0.
I20260812 16:37:06.401835 3681 raft_consensus.cc:2454] T c187fe17c9124d4d9a35d5ec761ce375 P d5d3d26c2a9b4f8c8f839739e0074511 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db07bd4898fb4d61854ec9f949702c2d in term 0.
I20260812 16:37:06.402015 4074 leader_election.cc:302] T c187fe17c9124d4d9a35d5ec761ce375 P db07bd4898fb4d61854ec9f949702c2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d5d3d26c2a9b4f8c8f839739e0074511, db07bd4898fb4d61854ec9f949702c2d; no voters:
I20260812 16:37:06.402146 4349 raft_consensus.cc:2781] T c187fe17c9124d4d9a35d5ec761ce375 P db07bd4898fb4d61854ec9f949702c2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:37:06.402196 4349 raft_consensus.cc:482] T c187fe17c9124d4d9a35d5ec761ce375 P db07bd4898fb4d61854ec9f949702c2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:37:06.402217 4349 raft_consensus.cc:3037] T c187fe17c9124d4d9a35d5ec761ce375 P db07bd4898fb4d61854ec9f949702c2d [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:06.402910 4349 raft_consensus.cc:504] T c187fe17c9124d4d9a35d5ec761ce375 P db07bd4898fb4d61854ec9f949702c2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 } } peers { permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 } } peers { permanent_uuid: "db07bd4898fb4d61854ec9f949702c2d" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32825 } }
I20260812 16:37:06.403064 4349 leader_election.cc:288] T c187fe17c9124d4d9a35d5ec761ce375 P db07bd4898fb4d61854ec9f949702c2d [CANDIDATE]: Term 1 election: Requested vote from peers d5d3d26c2a9b4f8c8f839739e0074511 (127.0.253.193:38327), 84c6eda51a614b22bc78dfe6da94141e (127.0.253.194:42549)
I20260812 16:37:06.403273 3681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c187fe17c9124d4d9a35d5ec761ce375" candidate_uuid: "db07bd4898fb4d61854ec9f949702c2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5d3d26c2a9b4f8c8f839739e0074511"
I20260812 16:37:06.403281 3939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c187fe17c9124d4d9a35d5ec761ce375" candidate_uuid: "db07bd4898fb4d61854ec9f949702c2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84c6eda51a614b22bc78dfe6da94141e"
I20260812 16:37:06.403375 3681 raft_consensus.cc:3037] T c187fe17c9124d4d9a35d5ec761ce375 P d5d3d26c2a9b4f8c8f839739e0074511 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:06.403374 3939 raft_consensus.cc:3037] T c187fe17c9124d4d9a35d5ec761ce375 P 84c6eda51a614b22bc78dfe6da94141e [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:06.404035 3939 raft_consensus.cc:2454] T c187fe17c9124d4d9a35d5ec761ce375 P 84c6eda51a614b22bc78dfe6da94141e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db07bd4898fb4d61854ec9f949702c2d in term 1.
I20260812 16:37:06.404075 3681 raft_consensus.cc:2454] T c187fe17c9124d4d9a35d5ec761ce375 P d5d3d26c2a9b4f8c8f839739e0074511 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db07bd4898fb4d61854ec9f949702c2d in term 1.
I20260812 16:37:06.404224 4077 leader_election.cc:302] T c187fe17c9124d4d9a35d5ec761ce375 P db07bd4898fb4d61854ec9f949702c2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84c6eda51a614b22bc78dfe6da94141e, db07bd4898fb4d61854ec9f949702c2d; no voters:
I20260812 16:37:06.404302 4349 raft_consensus.cc:2781] T c187fe17c9124d4d9a35d5ec761ce375 P db07bd4898fb4d61854ec9f949702c2d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:37:06.404342 4349 raft_consensus.cc:686] T c187fe17c9124d4d9a35d5ec761ce375 P db07bd4898fb4d61854ec9f949702c2d [term 1 LEADER]: Becoming Leader. State: Replica: db07bd4898fb4d61854ec9f949702c2d, State: Running, Role: LEADER
I20260812 16:37:06.404392 4349 consensus_queue.cc:237] T c187fe17c9124d4d9a35d5ec761ce375 P db07bd4898fb4d61854ec9f949702c2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 } } peers { permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 } } peers { permanent_uuid: "db07bd4898fb4d61854ec9f949702c2d" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32825 } }
I20260812 16:37:06.405025 3380 catalog_manager.cc:5495] T c187fe17c9124d4d9a35d5ec761ce375 P db07bd4898fb4d61854ec9f949702c2d reported cstate change: term changed from 0 to 1, leader changed from <none> to db07bd4898fb4d61854ec9f949702c2d (127.0.253.195). New cstate: current_term: 1 leader_uuid: "db07bd4898fb4d61854ec9f949702c2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db07bd4898fb4d61854ec9f949702c2d" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32825 } health_report { overall_health: HEALTHY } } }
Aug 12 16:37:06 dist-test-slave-w4v5 krb5kdc[3280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552625, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:06.435273 3461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:06.427955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55624 (local address 127.0.253.254:37355)
0812 16:37:06.428042 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:06.428045 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:06.428171 (+ 126us) server_negotiation.cc:407] Connection header received
0812 16:37:06.428257 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:06.428257 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:06.428271 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:06.428341 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:06.428840 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.429361 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.430123 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.430309 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.430404 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:06.432773 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:06.432775 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:06.432775 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:06.432782 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:06.434040 (+ 1258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:06.434453 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:06.434454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:06.434454 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:06.434510 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:06.434706 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:06.434706 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:06.434707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:06.434820 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:06.434892 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:06.435057 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 16:37:06.435163 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 16:37:06.435343 4400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:06.427862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:37355 (local address 127.0.0.1:55624)
0812 16:37:06.428074 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:06.428106 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:06.428194 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:06.428346 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:06.428346 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:06.428694 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:06.428749 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:06.428753 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.429459 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.429460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:06.430020 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:06.430021 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.430416 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.430418 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:06.430480 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:06.431037 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:06.431042 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:06.432675 (+ 1633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:06.434164 (+ 1489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:06.434172 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:06.434175 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:06.434367 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:06.434575 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:06.434576 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:06.434576 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:06.434641 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:06.434894 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:06.434902 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:06.434950 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 16:37:06.435032 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 16:37:06.435221 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":51,"thread_start_us":101,"threads_started":1}
I20260812 16:37:06.437515 3380 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:55624:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 16:37:06.437789 3380 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:37:06.439435 3904 tablet_service.cc:1467] Processing CreateTablet for tablet 7680a7f357cb43ad939de24024fab498 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5e11a7735e9c4247890d5ce3bc5f18b5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 16:37:06.439431 3645 tablet_service.cc:1467] Processing CreateTablet for tablet 7680a7f357cb43ad939de24024fab498 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5e11a7735e9c4247890d5ce3bc5f18b5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 16:37:06.439551 4159 tablet_service.cc:1467] Processing CreateTablet for tablet 7680a7f357cb43ad939de24024fab498 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5e11a7735e9c4247890d5ce3bc5f18b5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 16:37:06.439690 4159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7680a7f357cb43ad939de24024fab498. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:06.439695 3904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7680a7f357cb43ad939de24024fab498. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:06.439690 3645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7680a7f357cb43ad939de24024fab498. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:06.441164 4341 tablet_bootstrap.cc:492] T 7680a7f357cb43ad939de24024fab498 P 84c6eda51a614b22bc78dfe6da94141e: Bootstrap starting.
I20260812 16:37:06.441285 4337 tablet_bootstrap.cc:492] T 7680a7f357cb43ad939de24024fab498 P db07bd4898fb4d61854ec9f949702c2d: Bootstrap starting.
I20260812 16:37:06.441566 4341 tablet_bootstrap.cc:654] T 7680a7f357cb43ad939de24024fab498 P 84c6eda51a614b22bc78dfe6da94141e: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:06.441658 4337 tablet_bootstrap.cc:654] T 7680a7f357cb43ad939de24024fab498 P db07bd4898fb4d61854ec9f949702c2d: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:06.441946 4354 tablet_bootstrap.cc:492] T 7680a7f357cb43ad939de24024fab498 P d5d3d26c2a9b4f8c8f839739e0074511: Bootstrap starting.
I20260812 16:37:06.442245 4341 tablet_bootstrap.cc:492] T 7680a7f357cb43ad939de24024fab498 P 84c6eda51a614b22bc78dfe6da94141e: No bootstrap required, opened a new log
I20260812 16:37:06.442299 4341 ts_tablet_manager.cc:1397] T 7680a7f357cb43ad939de24024fab498 P 84c6eda51a614b22bc78dfe6da94141e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:06.442320 4337 tablet_bootstrap.cc:492] T 7680a7f357cb43ad939de24024fab498 P db07bd4898fb4d61854ec9f949702c2d: No bootstrap required, opened a new log
I20260812 16:37:06.442387 4337 ts_tablet_manager.cc:1397] T 7680a7f357cb43ad939de24024fab498 P db07bd4898fb4d61854ec9f949702c2d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:06.442428 4354 tablet_bootstrap.cc:654] T 7680a7f357cb43ad939de24024fab498 P d5d3d26c2a9b4f8c8f839739e0074511: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:06.442497 4341 raft_consensus.cc:348] T 7680a7f357cb43ad939de24024fab498 P 84c6eda51a614b22bc78dfe6da94141e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 } } peers { permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 } } peers { permanent_uuid: "db07bd4898fb4d61854ec9f949702c2d" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32825 } }
I20260812 16:37:06.442575 4341 raft_consensus.cc:374] T 7680a7f357cb43ad939de24024fab498 P 84c6eda51a614b22bc78dfe6da94141e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:06.442605 4341 raft_consensus.cc:729] T 7680a7f357cb43ad939de24024fab498 P 84c6eda51a614b22bc78dfe6da94141e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84c6eda51a614b22bc78dfe6da94141e, State: Initialized, Role: FOLLOWER
I20260812 16:37:06.442551 4337 raft_consensus.cc:348] T 7680a7f357cb43ad939de24024fab498 P db07bd4898fb4d61854ec9f949702c2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 } } peers { permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 } } peers { permanent_uuid: "db07bd4898fb4d61854ec9f949702c2d" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32825 } }
I20260812 16:37:06.442632 4337 raft_consensus.cc:374] T 7680a7f357cb43ad939de24024fab498 P db07bd4898fb4d61854ec9f949702c2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:06.442663 4337 raft_consensus.cc:729] T 7680a7f357cb43ad939de24024fab498 P db07bd4898fb4d61854ec9f949702c2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db07bd4898fb4d61854ec9f949702c2d, State: Initialized, Role: FOLLOWER
I20260812 16:37:06.442649 4341 consensus_queue.cc:260] T 7680a7f357cb43ad939de24024fab498 P 84c6eda51a614b22bc78dfe6da94141e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 } } peers { permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 } } peers { permanent_uuid: "db07bd4898fb4d61854ec9f949702c2d" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32825 } }
I20260812 16:37:06.442757 4341 ts_tablet_manager.cc:1428] T 7680a7f357cb43ad939de24024fab498 P 84c6eda51a614b22bc78dfe6da94141e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 16:37:06.442730 4337 consensus_queue.cc:260] T 7680a7f357cb43ad939de24024fab498 P db07bd4898fb4d61854ec9f949702c2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 } } peers { permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 } } peers { permanent_uuid: "db07bd4898fb4d61854ec9f949702c2d" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32825 } }
I20260812 16:37:06.442855 4337 ts_tablet_manager.cc:1428] T 7680a7f357cb43ad939de24024fab498 P db07bd4898fb4d61854ec9f949702c2d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:06.442926 4352 tablet_replica.cc:439] T 7680a7f357cb43ad939de24024fab498 P 84c6eda51a614b22bc78dfe6da94141e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 } } peers { permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 } } peers { permanent_uuid: "db07bd4898fb4d61854ec9f949702c2d" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32825 } } }
I20260812 16:37:06.443042 4352 tablet_replica.cc:442] T 7680a7f357cb43ad939de24024fab498 P 84c6eda51a614b22bc78dfe6da94141e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:37:06.443249 4354 tablet_bootstrap.cc:492] T 7680a7f357cb43ad939de24024fab498 P d5d3d26c2a9b4f8c8f839739e0074511: No bootstrap required, opened a new log
I20260812 16:37:06.443251 4349 tablet_replica.cc:439] T 7680a7f357cb43ad939de24024fab498 P db07bd4898fb4d61854ec9f949702c2d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 } } peers { permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 } } peers { permanent_uuid: "db07bd4898fb4d61854ec9f949702c2d" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32825 } } }
I20260812 16:37:06.443323 4354 ts_tablet_manager.cc:1397] T 7680a7f357cb43ad939de24024fab498 P d5d3d26c2a9b4f8c8f839739e0074511: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:06.443346 4349 tablet_replica.cc:442] T 7680a7f357cb43ad939de24024fab498 P db07bd4898fb4d61854ec9f949702c2d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:37:06.443500 4354 raft_consensus.cc:348] T 7680a7f357cb43ad939de24024fab498 P d5d3d26c2a9b4f8c8f839739e0074511 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 } } peers { permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 } } peers { permanent_uuid: "db07bd4898fb4d61854ec9f949702c2d" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32825 } }
I20260812 16:37:06.443579 4354 raft_consensus.cc:374] T 7680a7f357cb43ad939de24024fab498 P d5d3d26c2a9b4f8c8f839739e0074511 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:06.443604 4354 raft_consensus.cc:729] T 7680a7f357cb43ad939de24024fab498 P d5d3d26c2a9b4f8c8f839739e0074511 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5d3d26c2a9b4f8c8f839739e0074511, State: Initialized, Role: FOLLOWER
I20260812 16:37:06.443650 4354 consensus_queue.cc:260] T 7680a7f357cb43ad939de24024fab498 P d5d3d26c2a9b4f8c8f839739e0074511 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 } } peers { permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 } } peers { permanent_uuid: "db07bd4898fb4d61854ec9f949702c2d" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32825 } }
I20260812 16:37:06.443739 4354 ts_tablet_manager.cc:1428] T 7680a7f357cb43ad939de24024fab498 P d5d3d26c2a9b4f8c8f839739e0074511: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 16:37:06.443895 4356 tablet_replica.cc:439] T 7680a7f357cb43ad939de24024fab498 P d5d3d26c2a9b4f8c8f839739e0074511: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 } } peers { permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 } } peers { permanent_uuid: "db07bd4898fb4d61854ec9f949702c2d" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32825 } } }
I20260812 16:37:06.443990 4356 tablet_replica.cc:442] T 7680a7f357cb43ad939de24024fab498 P d5d3d26c2a9b4f8c8f839739e0074511: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:37:06.453204 4356 raft_consensus.cc:482] T 7680a7f357cb43ad939de24024fab498 P d5d3d26c2a9b4f8c8f839739e0074511 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:37:06.453302 4356 raft_consensus.cc:504] T 7680a7f357cb43ad939de24024fab498 P d5d3d26c2a9b4f8c8f839739e0074511 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 } } peers { permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 } } peers { permanent_uuid: "db07bd4898fb4d61854ec9f949702c2d" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32825 } }
I20260812 16:37:06.453657 4356 leader_election.cc:288] T 7680a7f357cb43ad939de24024fab498 P d5d3d26c2a9b4f8c8f839739e0074511 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84c6eda51a614b22bc78dfe6da94141e (127.0.253.194:42549), db07bd4898fb4d61854ec9f949702c2d (127.0.253.195:32825)
I20260812 16:37:06.457767 4335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:06.453875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:48063 (local address 127.0.253.194:42549)
0812 16:37:06.453939 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:06.453941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:06.454001 (+ 60us) server_negotiation.cc:407] Connection header received
0812 16:37:06.454153 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:06.454154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:06.454163 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:06.454236 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:06.454398 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.455165 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.456753 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.457307 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.457387 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:06.457397 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:06.457643 (+ 246us) server_negotiation.cc:299] Negotiation successful
0812 16:37:06.457687 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 16:37:06.457772 4331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:06.454067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:33543 (local address 127.0.253.195:32825)
0812 16:37:06.454105 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:06.454107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:06.454134 (+ 27us) server_negotiation.cc:407] Connection header received
0812 16:37:06.454206 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:06.454207 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:06.454220 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:06.454269 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:06.454398 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.455179 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.456753 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.457330 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.457391 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:06.457398 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:06.457654 (+ 256us) server_negotiation.cc:299] Negotiation successful
0812 16:37:06.457699 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 16:37:06.457831 4408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:06.453999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.195:32825 (local address 127.0.253.193:33543)
0812 16:37:06.454086 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:06.454094 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:06.454146 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:06.454269 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:06.454269 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:06.454270 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:06.454333 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:06.454334 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.455254 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.455255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:06.456673 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:06.456675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.457408 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.457410 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:06.457507 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:06.457513 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 16:37:06.457643 (+ 130us) client_negotiation.cc:240] Negotiation successful
0812 16:37:06.457700 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":50,"thread_start_us":42,"threads_started":1}
I20260812 16:37:06.458029 4405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:06.453800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:42549 (local address 127.0.253.193:48063)
0812 16:37:06.453929 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:06.453941 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:06.454024 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:06.454242 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:06.454243 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:06.454245 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:06.454315 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:06.454317 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.455254 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.455255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:06.456640 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:06.456644 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.457455 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.457456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:06.457575 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:06.457581 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 16:37:06.457637 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 16:37:06.457687 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":59,"thread_start_us":86,"threads_started":1}
I20260812 16:37:06.458031 4196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7680a7f357cb43ad939de24024fab498" candidate_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db07bd4898fb4d61854ec9f949702c2d" is_pre_election: true
I20260812 16:37:06.458189 4196 raft_consensus.cc:2454] T 7680a7f357cb43ad939de24024fab498 P db07bd4898fb4d61854ec9f949702c2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5d3d26c2a9b4f8c8f839739e0074511 in term 0.
I20260812 16:37:06.458258 3939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7680a7f357cb43ad939de24024fab498" candidate_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84c6eda51a614b22bc78dfe6da94141e" is_pre_election: true
I20260812 16:37:06.458338 3939 raft_consensus.cc:2454] T 7680a7f357cb43ad939de24024fab498 P 84c6eda51a614b22bc78dfe6da94141e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5d3d26c2a9b4f8c8f839739e0074511 in term 0.
I20260812 16:37:06.458370 3547 leader_election.cc:302] T 7680a7f357cb43ad939de24024fab498 P d5d3d26c2a9b4f8c8f839739e0074511 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d5d3d26c2a9b4f8c8f839739e0074511, db07bd4898fb4d61854ec9f949702c2d; no voters:
I20260812 16:37:06.458508 4356 raft_consensus.cc:2781] T 7680a7f357cb43ad939de24024fab498 P d5d3d26c2a9b4f8c8f839739e0074511 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:37:06.458554 4356 raft_consensus.cc:482] T 7680a7f357cb43ad939de24024fab498 P d5d3d26c2a9b4f8c8f839739e0074511 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:37:06.458583 4356 raft_consensus.cc:3037] T 7680a7f357cb43ad939de24024fab498 P d5d3d26c2a9b4f8c8f839739e0074511 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:06.459393 4356 raft_consensus.cc:504] T 7680a7f357cb43ad939de24024fab498 P d5d3d26c2a9b4f8c8f839739e0074511 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 } } peers { permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 } } peers { permanent_uuid: "db07bd4898fb4d61854ec9f949702c2d" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32825 } }
I20260812 16:37:06.459538 4356 leader_election.cc:288] T 7680a7f357cb43ad939de24024fab498 P d5d3d26c2a9b4f8c8f839739e0074511 [CANDIDATE]: Term 1 election: Requested vote from peers 84c6eda51a614b22bc78dfe6da94141e (127.0.253.194:42549), db07bd4898fb4d61854ec9f949702c2d (127.0.253.195:32825)
I20260812 16:37:06.459717 3939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7680a7f357cb43ad939de24024fab498" candidate_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84c6eda51a614b22bc78dfe6da94141e"
I20260812 16:37:06.459790 3939 raft_consensus.cc:3037] T 7680a7f357cb43ad939de24024fab498 P 84c6eda51a614b22bc78dfe6da94141e [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:06.459744 4196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7680a7f357cb43ad939de24024fab498" candidate_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db07bd4898fb4d61854ec9f949702c2d"
I20260812 16:37:06.459841 4196 raft_consensus.cc:3037] T 7680a7f357cb43ad939de24024fab498 P db07bd4898fb4d61854ec9f949702c2d [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:06.460373 3939 raft_consensus.cc:2454] T 7680a7f357cb43ad939de24024fab498 P 84c6eda51a614b22bc78dfe6da94141e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5d3d26c2a9b4f8c8f839739e0074511 in term 1.
I20260812 16:37:06.460537 3547 leader_election.cc:302] T 7680a7f357cb43ad939de24024fab498 P d5d3d26c2a9b4f8c8f839739e0074511 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84c6eda51a614b22bc78dfe6da94141e, d5d3d26c2a9b4f8c8f839739e0074511; no voters:
I20260812 16:37:06.460690 4196 raft_consensus.cc:2454] T 7680a7f357cb43ad939de24024fab498 P db07bd4898fb4d61854ec9f949702c2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5d3d26c2a9b4f8c8f839739e0074511 in term 1.
I20260812 16:37:06.460677 4356 raft_consensus.cc:2781] T 7680a7f357cb43ad939de24024fab498 P d5d3d26c2a9b4f8c8f839739e0074511 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:37:06.460888 4356 raft_consensus.cc:686] T 7680a7f357cb43ad939de24024fab498 P d5d3d26c2a9b4f8c8f839739e0074511 [term 1 LEADER]: Becoming Leader. State: Replica: d5d3d26c2a9b4f8c8f839739e0074511, State: Running, Role: LEADER
I20260812 16:37:06.460992 4356 consensus_queue.cc:237] T 7680a7f357cb43ad939de24024fab498 P d5d3d26c2a9b4f8c8f839739e0074511 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 } } peers { permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 } } peers { permanent_uuid: "db07bd4898fb4d61854ec9f949702c2d" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32825 } }
I20260812 16:37:06.461406 4412 tablet_replica.cc:439] T 7680a7f357cb43ad939de24024fab498 P d5d3d26c2a9b4f8c8f839739e0074511: TxnStatusTablet state changed. Reason: New leader d5d3d26c2a9b4f8c8f839739e0074511. Latest consensus state: current_term: 1 leader_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 } } peers { permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 } } peers { permanent_uuid: "db07bd4898fb4d61854ec9f949702c2d" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32825 } } }
I20260812 16:37:06.461508 4412 tablet_replica.cc:442] T 7680a7f357cb43ad939de24024fab498 P d5d3d26c2a9b4f8c8f839739e0074511: This TxnStatusTablet replica's current role is: LEADER
I20260812 16:37:06.461702 4416 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 16:37:06.461843 4416 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 16:37:06.461750 3378 catalog_manager.cc:5495] T 7680a7f357cb43ad939de24024fab498 P d5d3d26c2a9b4f8c8f839739e0074511 reported cstate change: term changed from 0 to 1, leader changed from <none> to d5d3d26c2a9b4f8c8f839739e0074511 (127.0.253.193). New cstate: current_term: 1 leader_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db07bd4898fb4d61854ec9f949702c2d" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32825 } health_report { overall_health: UNKNOWN } } }
I20260812 16:37:06.467767 4332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:06.464829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33720 (local address 127.0.253.193:38327)
0812 16:37:06.464869 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:06.464870 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:06.464978 (+ 108us) server_negotiation.cc:407] Connection header received
0812 16:37:06.465065 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:06.465065 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:06.465081 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:06.465138 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:37:06.465284 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.465762 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.466897 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.467137 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.467210 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:06.467431 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:37:06.467521 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:37:06.467571 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:06.467656 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 16:37:06.467676 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 16:37:06.467795 4400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:06.464791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:38327 (local address 127.0.0.1:33720)
0812 16:37:06.464896 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:06.464912 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:06.464987 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:06.465164 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:06.465165 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:06.465166 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 16:37:06.465219 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:06.465223 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.465845 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.465846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:06.466769 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:06.466772 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.467223 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.467225 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:06.467303 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:06.467323 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 16:37:06.467584 (+ 261us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 16:37:06.467594 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 16:37:06.467661 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 16:37:06.467717 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260812 16:37:06.468937 3939 raft_consensus.cc:1261] T 7680a7f357cb43ad939de24024fab498 P 84c6eda51a614b22bc78dfe6da94141e [term 1 FOLLOWER]: Refusing update from remote peer d5d3d26c2a9b4f8c8f839739e0074511: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:37:06.468986 4196 raft_consensus.cc:1261] T 7680a7f357cb43ad939de24024fab498 P db07bd4898fb4d61854ec9f949702c2d [term 1 FOLLOWER]: Refusing update from remote peer d5d3d26c2a9b4f8c8f839739e0074511: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:37:06.469296 4412 consensus_queue.cc:1035] T 7680a7f357cb43ad939de24024fab498 P d5d3d26c2a9b4f8c8f839739e0074511 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db07bd4898fb4d61854ec9f949702c2d" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:37:06.469404 4356 consensus_queue.cc:1035] T 7680a7f357cb43ad939de24024fab498 P d5d3d26c2a9b4f8c8f839739e0074511 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:37:06.470124 4349 tablet_replica.cc:439] T 7680a7f357cb43ad939de24024fab498 P db07bd4898fb4d61854ec9f949702c2d: TxnStatusTablet state changed. Reason: New leader d5d3d26c2a9b4f8c8f839739e0074511. Latest consensus state: current_term: 1 leader_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 } } peers { permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 } } peers { permanent_uuid: "db07bd4898fb4d61854ec9f949702c2d" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32825 } } }
I20260812 16:37:06.470227 4349 tablet_replica.cc:442] T 7680a7f357cb43ad939de24024fab498 P db07bd4898fb4d61854ec9f949702c2d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:37:06.470464 4352 tablet_replica.cc:439] T 7680a7f357cb43ad939de24024fab498 P 84c6eda51a614b22bc78dfe6da94141e: TxnStatusTablet state changed. Reason: New leader d5d3d26c2a9b4f8c8f839739e0074511. Latest consensus state: current_term: 1 leader_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 } } peers { permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 } } peers { permanent_uuid: "db07bd4898fb4d61854ec9f949702c2d" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32825 } } }
I20260812 16:37:06.470580 4352 tablet_replica.cc:442] T 7680a7f357cb43ad939de24024fab498 P 84c6eda51a614b22bc78dfe6da94141e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:37:06.471184 4356 tablet_replica.cc:439] T 7680a7f357cb43ad939de24024fab498 P d5d3d26c2a9b4f8c8f839739e0074511: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 } } peers { permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 } } peers { permanent_uuid: "db07bd4898fb4d61854ec9f949702c2d" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32825 } } }
I20260812 16:37:06.471199 4424 tablet_replica.cc:439] T 7680a7f357cb43ad939de24024fab498 P d5d3d26c2a9b4f8c8f839739e0074511: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 } } peers { permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 } } peers { permanent_uuid: "db07bd4898fb4d61854ec9f949702c2d" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32825 } } }
I20260812 16:37:06.471310 4356 tablet_replica.cc:442] T 7680a7f357cb43ad939de24024fab498 P d5d3d26c2a9b4f8c8f839739e0074511: This TxnStatusTablet replica's current role is: LEADER
I20260812 16:37:06.471323 4424 tablet_replica.cc:442] T 7680a7f357cb43ad939de24024fab498 P d5d3d26c2a9b4f8c8f839739e0074511: This TxnStatusTablet replica's current role is: LEADER
I20260812 16:37:06.471668 4349 tablet_replica.cc:439] T 7680a7f357cb43ad939de24024fab498 P db07bd4898fb4d61854ec9f949702c2d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 } } peers { permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 } } peers { permanent_uuid: "db07bd4898fb4d61854ec9f949702c2d" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32825 } } }
I20260812 16:37:06.471752 4349 tablet_replica.cc:442] T 7680a7f357cb43ad939de24024fab498 P db07bd4898fb4d61854ec9f949702c2d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:37:06.471884 4363 mvcc.cc:205] Tried to move back new op lower bound from 7317719558014615552 to 7317719557984952320. Current Snapshot: MvccSnapshot[applied={T|T < 7317719558014615552 or (T in {7317719558014615552})}]
I20260812 16:37:06.472930 4352 tablet_replica.cc:439] T 7680a7f357cb43ad939de24024fab498 P 84c6eda51a614b22bc78dfe6da94141e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 } } peers { permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 } } peers { permanent_uuid: "db07bd4898fb4d61854ec9f949702c2d" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32825 } } }
I20260812 16:37:06.473016 4352 tablet_replica.cc:442] T 7680a7f357cb43ad939de24024fab498 P 84c6eda51a614b22bc78dfe6da94141e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:37:06.478597 4332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:06.476069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:46475 (local address 127.0.253.193:38327)
0812 16:37:06.476123 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:06.476125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:06.476132 (+ 7us) server_negotiation.cc:407] Connection header received
0812 16:37:06.476150 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:06.476151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:06.476158 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:06.476216 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:37:06.476509 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.476944 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.478027 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.478193 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.478246 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:06.478426 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:37:06.478450 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:37:06.478487 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:06.478540 (+ 53us) server_negotiation.cc:299] Negotiation successful
0812 16:37:06.478554 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 16:37:06.478668 4299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:06.475850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:38327 (local address 127.0.253.195:46475)
0812 16:37:06.475937 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:06.475950 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:06.475994 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:06.476232 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:06.476233 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:06.476234 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 16:37:06.476282 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:06.476283 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.477019 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.477020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:06.477944 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:06.477946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.478269 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.478270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:06.478332 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:06.478349 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 16:37:06.478489 (+ 140us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 16:37:06.478490 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 16:37:06.478539 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 16:37:06.478585 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 16:37:06.480785 3939 raft_consensus.cc:1261] T c187fe17c9124d4d9a35d5ec761ce375 P 84c6eda51a614b22bc78dfe6da94141e [term 1 FOLLOWER]: Refusing update from remote peer db07bd4898fb4d61854ec9f949702c2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:37:06.480810 3681 raft_consensus.cc:1261] T c187fe17c9124d4d9a35d5ec761ce375 P d5d3d26c2a9b4f8c8f839739e0074511 [term 1 FOLLOWER]: Refusing update from remote peer db07bd4898fb4d61854ec9f949702c2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:37:06.480991 4358 consensus_queue.cc:1035] T c187fe17c9124d4d9a35d5ec761ce375 P db07bd4898fb4d61854ec9f949702c2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c6eda51a614b22bc78dfe6da94141e" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:37:06.481107 4349 consensus_queue.cc:1035] T c187fe17c9124d4d9a35d5ec761ce375 P db07bd4898fb4d61854ec9f949702c2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5d3d26c2a9b4f8c8f839739e0074511" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 38327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:37:06.482998 4372 mvcc.cc:205] Tried to move back new op lower bound from 7317719558063792128 to 7317719557752643584. Current Snapshot: MvccSnapshot[applied={T|T < 7317719558063792128}]
I20260812 16:37:06.490509 4331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:06.487743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:36955 (local address 127.0.253.195:32825)
0812 16:37:06.487808 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:06.487811 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:06.487888 (+ 77us) server_negotiation.cc:407] Connection header received
0812 16:37:06.487952 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:06.487952 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:06.487961 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:06.488003 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:37:06.488134 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.488583 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.489806 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.489985 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.490036 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:06.490262 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:37:06.490296 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:37:06.490345 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:06.490426 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 16:37:06.490446 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 16:37:06.490576 4433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:06.487562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.195:32825 (local address 127.0.253.193:36955)
0812 16:37:06.487827 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:06.487839 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:06.487899 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:06.488006 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:06.488007 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:06.488008 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 16:37:06.488074 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:06.488076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.488700 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.488701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:06.489713 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:06.489716 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.490076 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.490077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:06.490150 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:06.490168 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 16:37:06.490366 (+ 198us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 16:37:06.490368 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 16:37:06.490422 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 16:37:06.490467 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":80,"threads_started":1}
I20260812 16:37:06.499169 3378 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55608:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:37:06.499270 3378 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55608:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:37:06.499914 3378 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 80e95251a626458bb717456269c4ea24: Sending DeleteTablet for 3 replicas of tablet c187fe17c9124d4d9a35d5ec761ce375
I20260812 16:37:06.500188 4159 tablet_service.cc:1514] Processing DeleteTablet for tablet c187fe17c9124d4d9a35d5ec761ce375 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:06 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:46592
I20260812 16:37:06.500203 3904 tablet_service.cc:1514] Processing DeleteTablet for tablet c187fe17c9124d4d9a35d5ec761ce375 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:06 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:55480
I20260812 16:37:06.500298 4383 tablet_replica.cc:330] T c187fe17c9124d4d9a35d5ec761ce375 P 84c6eda51a614b22bc78dfe6da94141e: stopping tablet replica
I20260812 16:37:06.500303 4386 tablet_replica.cc:330] T c187fe17c9124d4d9a35d5ec761ce375 P db07bd4898fb4d61854ec9f949702c2d: stopping tablet replica
I20260812 16:37:06.500348 4386 raft_consensus.cc:2229] T c187fe17c9124d4d9a35d5ec761ce375 P db07bd4898fb4d61854ec9f949702c2d [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:37:06.500348 4383 raft_consensus.cc:2229] T c187fe17c9124d4d9a35d5ec761ce375 P 84c6eda51a614b22bc78dfe6da94141e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:37:06.500383 4383 raft_consensus.cc:2258] T c187fe17c9124d4d9a35d5ec761ce375 P 84c6eda51a614b22bc78dfe6da94141e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:06.500396 4386 raft_consensus.cc:2258] T c187fe17c9124d4d9a35d5ec761ce375 P db07bd4898fb4d61854ec9f949702c2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:06.500725 4386 ts_tablet_manager.cc:1905] T c187fe17c9124d4d9a35d5ec761ce375 P db07bd4898fb4d61854ec9f949702c2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:37:06.500718 4383 ts_tablet_manager.cc:1905] T c187fe17c9124d4d9a35d5ec761ce375 P 84c6eda51a614b22bc78dfe6da94141e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:37:06.502110 3645 tablet_service.cc:1514] Processing DeleteTablet for tablet c187fe17c9124d4d9a35d5ec761ce375 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:06 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:33718
I20260812 16:37:06.502204 4380 tablet_replica.cc:330] T c187fe17c9124d4d9a35d5ec761ce375 P d5d3d26c2a9b4f8c8f839739e0074511: stopping tablet replica
I20260812 16:37:06.502242 4380 raft_consensus.cc:2229] T c187fe17c9124d4d9a35d5ec761ce375 P d5d3d26c2a9b4f8c8f839739e0074511 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:37:06.502274 4380 raft_consensus.cc:2258] T c187fe17c9124d4d9a35d5ec761ce375 P d5d3d26c2a9b4f8c8f839739e0074511 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:06.502405 4383 ts_tablet_manager.cc:1918] T c187fe17c9124d4d9a35d5ec761ce375 P 84c6eda51a614b22bc78dfe6da94141e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:37:06.502458 4383 log.cc:1196] T c187fe17c9124d4d9a35d5ec761ce375 P 84c6eda51a614b22bc78dfe6da94141e: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-1/wal/wals/c187fe17c9124d4d9a35d5ec761ce375
I20260812 16:37:06.502584 4380 ts_tablet_manager.cc:1905] T c187fe17c9124d4d9a35d5ec761ce375 P d5d3d26c2a9b4f8c8f839739e0074511: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:37:06.502691 4383 ts_tablet_manager.cc:1939] T c187fe17c9124d4d9a35d5ec761ce375 P 84c6eda51a614b22bc78dfe6da94141e: Deleting consensus metadata
I20260812 16:37:06.502995 3359 catalog_manager.cc:4841] TS 84c6eda51a614b22bc78dfe6da94141e (127.0.253.194:42549): tablet c187fe17c9124d4d9a35d5ec761ce375 (table test-table [id=164bbe7feb524cf390c3f996ed717c65]) successfully deleted
I20260812 16:37:06.503038 4386 ts_tablet_manager.cc:1918] T c187fe17c9124d4d9a35d5ec761ce375 P db07bd4898fb4d61854ec9f949702c2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:37:06.503182 4386 log.cc:1196] T c187fe17c9124d4d9a35d5ec761ce375 P db07bd4898fb4d61854ec9f949702c2d: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-2/wal/wals/c187fe17c9124d4d9a35d5ec761ce375
I20260812 16:37:06.503443 4386 ts_tablet_manager.cc:1939] T c187fe17c9124d4d9a35d5ec761ce375 P db07bd4898fb4d61854ec9f949702c2d: Deleting consensus metadata
I20260812 16:37:06.503763 3359 catalog_manager.cc:4841] TS db07bd4898fb4d61854ec9f949702c2d (127.0.253.195:32825): tablet c187fe17c9124d4d9a35d5ec761ce375 (table test-table [id=164bbe7feb524cf390c3f996ed717c65]) successfully deleted
I20260812 16:37:06.504324 4380 ts_tablet_manager.cc:1918] T c187fe17c9124d4d9a35d5ec761ce375 P d5d3d26c2a9b4f8c8f839739e0074511: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:37:06.504402 4380 log.cc:1196] T c187fe17c9124d4d9a35d5ec761ce375 P d5d3d26c2a9b4f8c8f839739e0074511: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552619305671-1015-0/minicluster-data/ts-0/wal/wals/c187fe17c9124d4d9a35d5ec761ce375
I20260812 16:37:06.504602 4380 ts_tablet_manager.cc:1939] T c187fe17c9124d4d9a35d5ec761ce375 P d5d3d26c2a9b4f8c8f839739e0074511: Deleting consensus metadata
I20260812 16:37:06.504935 3355 catalog_manager.cc:4841] TS d5d3d26c2a9b4f8c8f839739e0074511 (127.0.253.193:38327): tablet c187fe17c9124d4d9a35d5ec761ce375 (table test-table [id=164bbe7feb524cf390c3f996ed717c65]) successfully deleted
Aug 12 16:37:06 dist-test-slave-w4v5 krb5kdc[3280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552626, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.253.193@KRBTEST.COM
I20260812 16:37:06.519169 4332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:06.503288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33726 (local address 127.0.253.193:38327)
0812 16:37:06.503839 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:06.503841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:06.510563 (+ 6722us) server_negotiation.cc:407] Connection header received
0812 16:37:06.510649 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:06.510650 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:06.510666 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:06.510764 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:06.511427 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.511895 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.512713 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.512897 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.512973 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:06.515951 (+ 2978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:06.515960 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:06.515961 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:06.515968 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:06.517902 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:06.518286 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:06.518290 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:06.518291 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:06.518359 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:06.518586 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:06.518587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:06.518588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:06.518749 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:06.518903 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:06.518952 (+ 49us) server_negotiation.cc:299] Negotiation successful
0812 16:37:06.519030 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526}
W20260812 16:37:06.519555 3762 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33726
I20260812 16:37:06.527232 3461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:06.520651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55630 (local address 127.0.253.254:37355)
0812 16:37:06.520717 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:06.520719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:06.520784 (+ 65us) server_negotiation.cc:407] Connection header received
0812 16:37:06.520818 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:06.520820 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:06.520831 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:06.520902 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:06.521350 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.522156 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.522837 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:06.523173 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:06.523239 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:06.524404 (+ 1165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:06.524405 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:06.524406 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:06.524414 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:06.526068 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:06.526398 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:06.526399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:06.526400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:06.526460 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:06.526646 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:06.526646 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:06.526647 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:06.526783 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:06.526888 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:06.527033 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 16:37:06.527135 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260812 16:37:06.527432 3378 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55630
I20260812 16:37:06.528162 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 3510
I20260812 16:37:06.549281 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 3784
I20260812 16:37:06.558465 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 4046
I20260812 16:37:06.573283 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 3306
2026-08-12T16:37:06Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2608 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:37:06 dist-test-slave-w4v5 krb5kdc[4447](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:37:06 dist-test-slave-w4v5 krb5kdc[4447](info): set up 2 sockets
Aug 12 16:37:06 dist-test-slave-w4v5 krb5kdc[4447](info): commencing operation
krb5kdc: starting...
W20260812 16:37:07.903362 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.296s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:37:07 dist-test-slave-w4v5 krb5kdc[4447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:37:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:37:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
I20260812 16:37:08.030217 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:41125
--webserver_interface=127.0.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:33653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:41125
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:08.113104 4466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:08.113363 4466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:08.113401 4466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:08.114527 4466 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:37:08.114604 4466 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 16:37:08.114626 4466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:08.114663 4466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:08.114696 4466 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:37:08.114718 4466 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:37:08.116202 4466 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:33653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:41125
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:41125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:08.116497 4466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:08.116775 4466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:08.122046 4474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:08.122046 4480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:08.122074 4475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:08.122372 4466 server_base.cc:1034] running on GCE node
I20260812 16:37:08.122701 4466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:08.123000 4466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:08.124181 4466 hybrid_clock.cc:648] HybridClock initialized: now 1786552628124164 us; error 47 us; skew 500 ppm
Aug 12 16:37:08 dist-test-slave-w4v5 krb5kdc[4447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552628, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:08.126763 4466 init.cc:377] Logged in from keytab as kudu/127.0.253.254@KRBTEST.COM (short username kudu)
I20260812 16:37:08.127225 4466 webserver.cc:460] Webserver started at http://127.0.253.254:39329/ using document root <none> and password file <none>
I20260812 16:37:08.127481 4466 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:08.127552 4466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:08.127702 4466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:08.128753 4466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "028fd69b0c7645bca0e07c1476188e82"
format_stamp: "Formatted at 2026-08-12 16:37:08 on dist-test-slave-w4v5"
server_key: "9fed0b8844c2d112d09ca254ad74ecc8"
server_key_iv: "b51553036159043af8ac1de13d833f35"
server_key_version: "encryptionkey@0"
I20260812 16:37:08.129204 4466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "028fd69b0c7645bca0e07c1476188e82"
format_stamp: "Formatted at 2026-08-12 16:37:08 on dist-test-slave-w4v5"
server_key: "9fed0b8844c2d112d09ca254ad74ecc8"
server_key_iv: "b51553036159043af8ac1de13d833f35"
server_key_version: "encryptionkey@0"
I20260812 16:37:08.130607 4466 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:37:08.131385 4491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:08.131562 4466 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:08.131673 4466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "028fd69b0c7645bca0e07c1476188e82"
format_stamp: "Formatted at 2026-08-12 16:37:08 on dist-test-slave-w4v5"
server_key: "9fed0b8844c2d112d09ca254ad74ecc8"
server_key_iv: "b51553036159043af8ac1de13d833f35"
server_key_version: "encryptionkey@0"
I20260812 16:37:08.131750 4466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:08.158872 4466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:08.161372 4466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:08.161547 4466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:08.166517 4466 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:41125
I20260812 16:37:08.166613 4620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:41125 every 8 connection(s)
I20260812 16:37:08.166945 4466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:37:08.167778 4621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:08.170152 4621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 028fd69b0c7645bca0e07c1476188e82: Bootstrap starting.
I20260812 16:37:08.170802 4621 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 028fd69b0c7645bca0e07c1476188e82: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:08.171164 4621 log.cc:824] T 00000000000000000000000000000000 P 028fd69b0c7645bca0e07c1476188e82: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:08.171962 4621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 028fd69b0c7645bca0e07c1476188e82: No bootstrap required, opened a new log
I20260812 16:37:08.174311 4621 raft_consensus.cc:348] T 00000000000000000000000000000000 P 028fd69b0c7645bca0e07c1476188e82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "028fd69b0c7645bca0e07c1476188e82" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 41125 } }
I20260812 16:37:08.174458 4621 raft_consensus.cc:374] T 00000000000000000000000000000000 P 028fd69b0c7645bca0e07c1476188e82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:08.174480 4621 raft_consensus.cc:729] T 00000000000000000000000000000000 P 028fd69b0c7645bca0e07c1476188e82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 028fd69b0c7645bca0e07c1476188e82, State: Initialized, Role: FOLLOWER
I20260812 16:37:08.174643 4621 consensus_queue.cc:260] T 00000000000000000000000000000000 P 028fd69b0c7645bca0e07c1476188e82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "028fd69b0c7645bca0e07c1476188e82" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 41125 } }
I20260812 16:37:08.174728 4621 raft_consensus.cc:388] T 00000000000000000000000000000000 P 028fd69b0c7645bca0e07c1476188e82 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:37:08.174752 4621 raft_consensus.cc:482] T 00000000000000000000000000000000 P 028fd69b0c7645bca0e07c1476188e82 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:37:08.174817 4621 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 028fd69b0c7645bca0e07c1476188e82 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:08.175457 4621 raft_consensus.cc:504] T 00000000000000000000000000000000 P 028fd69b0c7645bca0e07c1476188e82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "028fd69b0c7645bca0e07c1476188e82" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 41125 } }
I20260812 16:37:08.175580 4621 leader_election.cc:302] T 00000000000000000000000000000000 P 028fd69b0c7645bca0e07c1476188e82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 028fd69b0c7645bca0e07c1476188e82; no voters:
I20260812 16:37:08.175777 4621 leader_election.cc:288] T 00000000000000000000000000000000 P 028fd69b0c7645bca0e07c1476188e82 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:37:08.175938 4628 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 028fd69b0c7645bca0e07c1476188e82 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:37:08.175952 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 4466
I20260812 16:37:08.176036 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:37:08.176047 4621 sys_catalog.cc:564] T 00000000000000000000000000000000 P 028fd69b0c7645bca0e07c1476188e82 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:37:08.176198 4628 raft_consensus.cc:686] T 00000000000000000000000000000000 P 028fd69b0c7645bca0e07c1476188e82 [term 1 LEADER]: Becoming Leader. State: Replica: 028fd69b0c7645bca0e07c1476188e82, State: Running, Role: LEADER
I20260812 16:37:08.176244 1015 external_mini_cluster.cc:1428] Setting key b5c721a26ee8fb38fab6887e875ec6e2
I20260812 16:37:08.176299 4628 consensus_queue.cc:237] T 00000000000000000000000000000000 P 028fd69b0c7645bca0e07c1476188e82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "028fd69b0c7645bca0e07c1476188e82" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 41125 } }
I20260812 16:37:08.176713 4629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 028fd69b0c7645bca0e07c1476188e82 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "028fd69b0c7645bca0e07c1476188e82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "028fd69b0c7645bca0e07c1476188e82" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 41125 } } }
I20260812 16:37:08.176810 4629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 028fd69b0c7645bca0e07c1476188e82 [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:08.177062 4630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 028fd69b0c7645bca0e07c1476188e82 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 028fd69b0c7645bca0e07c1476188e82. Latest consensus state: current_term: 1 leader_uuid: "028fd69b0c7645bca0e07c1476188e82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "028fd69b0c7645bca0e07c1476188e82" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 41125 } } }
I20260812 16:37:08.177155 4630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 028fd69b0c7645bca0e07c1476188e82 [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:08.177348 4640 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:37:08.178105 4640 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:37:08.180392 4640 catalog_manager.cc:1347] Generated new cluster ID: f7eff69ac82140faafcd4600638b74a0
I20260812 16:37:08.180446 4640 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 16:37:08 dist-test-slave-w4v5 krb5kdc[4447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:08.188453 4642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:08.176817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38374 (local address 127.0.253.254:41125)
0812 16:37:08.177047 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:08.177056 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:08.177090 (+ 34us) server_negotiation.cc:407] Connection header received
0812 16:37:08.177214 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:08.177221 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:08.177292 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:08.178365 (+ 1073us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:08.178464 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.179892 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.180957 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.181583 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.181709 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:08.184617 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:08.184622 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:08.184622 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:08.184635 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:08.186670 (+ 2035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:08.187123 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:08.187126 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:08.187127 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:08.187256 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:08.187486 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:08.187487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:08.187487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:08.187800 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:08.187971 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:08.188082 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 16:37:08.188232 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":101,"threads_started":1}
I20260812 16:37:08.196084 4640 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:37:08.196617 4640 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:37:08.198616 4640 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 028fd69b0c7645bca0e07c1476188e82: Generated new TSK 0
I20260812 16:37:08.198854 4640 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab.
Entry for principal kudu/127.0.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab.
I20260812 16:37:08.233604 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.193:0
--local_ip_for_outbound_sockets=127.0.253.193
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:41125
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:33653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:08.331060 4662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:08.331321 4662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:08.331346 4662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:08.332484 4662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:08.332525 4662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:08.332544 4662 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 16:37:08.332592 4662 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 16:37:08.332618 4662 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.193
I20260812 16:37:08.334169 4662 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:33653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.253.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.253.254:41125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.193
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:08.334419 4662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:08.334681 4662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:08.340022 4676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:08.340037 4680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:08.340041 4673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:08.340374 4662 server_base.cc:1034] running on GCE node
I20260812 16:37:08.340543 4662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:08.340739 4662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:08.341878 4662 hybrid_clock.cc:648] HybridClock initialized: now 1786552628341867 us; error 26 us; skew 500 ppm
Aug 12 16:37:08 dist-test-slave-w4v5 krb5kdc[4447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552628, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:08.344460 4662 init.cc:377] Logged in from keytab as kudu/127.0.253.193@KRBTEST.COM (short username kudu)
I20260812 16:37:08.344942 4662 webserver.cc:460] Webserver started at http://127.0.253.193:41957/ using document root <none> and password file <none>
I20260812 16:37:08.345196 4662 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:08.345242 4662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:08.345405 4662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:08.346439 4662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-0/data/instance:
uuid: "b36bcb39624f44849c3b9ccfadfbf34d"
format_stamp: "Formatted at 2026-08-12 16:37:08 on dist-test-slave-w4v5"
server_key: "bf9dff498e74fadc7ae0812e56840715"
server_key_iv: "db8be2dd7abc597fc123bb007cd0f1f7"
server_key_version: "encryptionkey@0"
I20260812 16:37:08.346801 4662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance:
uuid: "b36bcb39624f44849c3b9ccfadfbf34d"
format_stamp: "Formatted at 2026-08-12 16:37:08 on dist-test-slave-w4v5"
server_key: "bf9dff498e74fadc7ae0812e56840715"
server_key_iv: "db8be2dd7abc597fc123bb007cd0f1f7"
server_key_version: "encryptionkey@0"
I20260812 16:37:08.348260 4662 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 16:37:08.349093 4691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:08.349294 4662 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 16:37:08.349431 4662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-0/wal
uuid: "b36bcb39624f44849c3b9ccfadfbf34d"
format_stamp: "Formatted at 2026-08-12 16:37:08 on dist-test-slave-w4v5"
server_key: "bf9dff498e74fadc7ae0812e56840715"
server_key_iv: "db8be2dd7abc597fc123bb007cd0f1f7"
server_key_version: "encryptionkey@0"
I20260812 16:37:08.349535 4662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:08.369204 4662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:08.371420 4662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:08.371573 4662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:08.372144 4662 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:08.372182 4662 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:08.372205 4662 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:08.372218 4662 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 16:37:08 dist-test-slave-w4v5 krb5kdc[4447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552628, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:08.379977 4662 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.193:46443
I20260812 16:37:08.380064 4916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.193:46443 every 8 connection(s)
I20260812 16:37:08.380394 4662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
I20260812 16:37:08.382908 4642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:08.372776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:39827 (local address 127.0.253.254:41125)
0812 16:37:08.372834 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:08.372836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:08.373084 (+ 248us) server_negotiation.cc:407] Connection header received
0812 16:37:08.373576 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:08.373577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:08.373596 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:08.373674 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:08.374604 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.375492 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.376679 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.377058 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.377147 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:08.380276 (+ 3129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:08.380277 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:08.380278 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:08.380286 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:08.381702 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:08.382072 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:08.382073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:08.382073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:08.382146 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:08.382420 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:08.382420 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:08.382421 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:08.382525 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:08.382575 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:08.382780 (+ 205us) server_negotiation.cc:299] Negotiation successful
0812 16:37:08.382846 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 16:37:08.383198 4709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:08.372752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41125 (local address 127.0.253.193:39827)
0812 16:37:08.372949 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:08.372985 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:08.373469 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:08.373772 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:08.373776 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:08.374166 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:08.374478 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:08.374485 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.375585 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.375586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:08.376576 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:08.376578 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.377251 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.377252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:08.377433 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:08.378027 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:08.378037 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:08.380185 (+ 2148us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:08.381777 (+ 1592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:08.381783 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:08.381785 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:08.382003 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:08.382237 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:08.382238 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:08.382238 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:08.382352 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:08.382566 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:08.382569 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:08.382689 (+ 120us) client_negotiation.cc:769] Sending connection context
0812 16:37:08.382767 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 16:37:08.382894 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":104,"threads_started":1}
I20260812 16:37:08.383759 4917 heartbeater.cc:347] Connected to a master server at 127.0.253.254:41125
I20260812 16:37:08.383845 4917 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:08.384042 4917 heartbeater.cc:511] Master 127.0.253.254:41125 requested a full tablet report, sending...
I20260812 16:37:08.384610 4526 ts_manager.cc:194] Registered new tserver with Master: b36bcb39624f44849c3b9ccfadfbf34d (127.0.253.193:46443)
I20260812 16:37:08.385568 4526 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.193@KRBTEST.COM'} at 127.0.253.193:39827
I20260812 16:37:08.389993 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 4662
I20260812 16:37:08.390076 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance
I20260812 16:37:08.390280 1015 external_mini_cluster.cc:1428] Setting key 95b7d563a45ed0f650caab047cae2d3f
I20260812 16:37:08.391203 4642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:08.384972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:39327 (local address 127.0.253.254:41125)
0812 16:37:08.385018 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:08.385020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:08.385120 (+ 100us) server_negotiation.cc:407] Connection header received
0812 16:37:08.385187 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:08.385188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:08.385197 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:08.385239 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:08.385600 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.386175 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.386947 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.387197 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.387282 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:08.388053 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:08.388054 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:08.388054 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:08.388060 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:08.389466 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:08.389776 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:08.389777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:08.389777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:08.389822 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:08.390577 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:08.390578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:08.390578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:08.390705 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:08.390771 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:08.390989 (+ 218us) server_negotiation.cc:299] Negotiation successful
0812 16:37:08.391114 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 16:37:08.391211 4924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:08.384892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41125 (local address 127.0.253.193:39327)
0812 16:37:08.385061 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:08.385070 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:08.385132 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:08.385251 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:08.385252 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:08.385427 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:08.385495 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:08.385497 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.386264 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.386265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:08.386873 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:08.386875 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.387281 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.387281 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:08.387334 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:08.387691 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:08.387695 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:08.387976 (+ 281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:08.389543 (+ 1567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:08.389544 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:08.389544 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:08.389717 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:08.390393 (+ 676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:08.390394 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:08.390394 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:08.390466 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:08.390776 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:08.390777 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:08.390859 (+ 82us) client_negotiation.cc:769] Sending connection context
0812 16:37:08.390936 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 16:37:08.391060 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":64,"threads_started":1}
WARNING: no policy specified for kudu/127.0.253.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab.
Entry for principal kudu/127.0.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab.
I20260812 16:37:08.418465 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.194:0
--local_ip_for_outbound_sockets=127.0.253.194
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:41125
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:33653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:08.501415 4927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:08.501690 4927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:08.501732 4927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:08.502887 4927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:08.502971 4927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:08.503005 4927 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 16:37:08.503032 4927 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 16:37:08.503106 4927 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.194
I20260812 16:37:08.504606 4927 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:33653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.253.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.253.254:41125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4927
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.194
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:08.504887 4927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:08.505123 4927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:08.510456 4945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:08.510514 4938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:08.510464 4939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:08.510861 4927 server_base.cc:1034] running on GCE node
I20260812 16:37:08.511133 4927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:08.511386 4927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:08.512575 4927 hybrid_clock.cc:648] HybridClock initialized: now 1786552628512562 us; error 44 us; skew 500 ppm
Aug 12 16:37:08 dist-test-slave-w4v5 krb5kdc[4447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552628, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:08.518985 4927 init.cc:377] Logged in from keytab as kudu/127.0.253.194@KRBTEST.COM (short username kudu)
I20260812 16:37:08.519613 4927 webserver.cc:460] Webserver started at http://127.0.253.194:32895/ using document root <none> and password file <none>
I20260812 16:37:08.519882 4927 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:08.519955 4927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:08.520112 4927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:08.521240 4927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-1/data/instance:
uuid: "7133565a63d24fe09d46b0f58def254b"
format_stamp: "Formatted at 2026-08-12 16:37:08 on dist-test-slave-w4v5"
server_key: "25a04a1e70660380559edf7218240ba7"
server_key_iv: "ab08f0ec121fae374aaa180643ddea30"
server_key_version: "encryptionkey@0"
I20260812 16:37:08.521622 4927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance:
uuid: "7133565a63d24fe09d46b0f58def254b"
format_stamp: "Formatted at 2026-08-12 16:37:08 on dist-test-slave-w4v5"
server_key: "25a04a1e70660380559edf7218240ba7"
server_key_iv: "ab08f0ec121fae374aaa180643ddea30"
server_key_version: "encryptionkey@0"
I20260812 16:37:08.523031 4927 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:37:08.523960 4957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:08.524166 4927 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:08.524289 4927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-1/wal
uuid: "7133565a63d24fe09d46b0f58def254b"
format_stamp: "Formatted at 2026-08-12 16:37:08 on dist-test-slave-w4v5"
server_key: "25a04a1e70660380559edf7218240ba7"
server_key_iv: "ab08f0ec121fae374aaa180643ddea30"
server_key_version: "encryptionkey@0"
I20260812 16:37:08.524366 4927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:08.550133 4927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:08.552394 4927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:08.552570 4927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:08.553184 4927 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:08.553246 4927 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:08.553287 4927 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:08.553308 4927 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 16:37:08 dist-test-slave-w4v5 krb5kdc[4447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552628, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:08.560801 4927 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.194:37559
I20260812 16:37:08.560952 5164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.194:37559 every 8 connection(s)
I20260812 16:37:08.561229 4927 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
I20260812 16:37:08.564255 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 4927
I20260812 16:37:08.564339 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance
I20260812 16:37:08.564544 1015 external_mini_cluster.cc:1428] Setting key 0f8a60345a4c29aa7fb4f558320e218d
I20260812 16:37:08.567298 4642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:08.553900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:40559 (local address 127.0.253.254:41125)
0812 16:37:08.553977 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:08.553979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:08.553991 (+ 12us) server_negotiation.cc:407] Connection header received
0812 16:37:08.554712 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:08.554713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:08.554749 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:08.554813 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:08.555839 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.556363 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.558059 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.558264 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.558373 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:08.561330 (+ 2957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:08.561332 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:08.561332 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:08.561341 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:08.565751 (+ 4410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:08.566190 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:08.566192 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:08.566192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:08.566264 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:08.566571 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:08.566572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:08.566572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:08.566699 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:08.566774 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:08.567119 (+ 345us) server_negotiation.cc:299] Negotiation successful
0812 16:37:08.567210 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 16:37:08.567493 4968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:08.553606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41125 (local address 127.0.253.194:40559)
0812 16:37:08.553789 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:08.553843 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:08.554568 (+ 725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:08.554896 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:08.554901 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:08.555341 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:08.555677 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:08.555685 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.556481 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.556482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:08.557951 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:08.557954 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.558398 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.558400 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:08.558564 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:08.559650 (+ 1086us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:08.559659 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:08.561216 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:08.565843 (+ 4627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:08.565852 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:08.565855 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:08.566093 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:08.566337 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:08.566338 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:08.566339 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:08.566484 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:08.566776 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:08.566784 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:08.566935 (+ 151us) client_negotiation.cc:769] Sending connection context
0812 16:37:08.567047 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 16:37:08.567247 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":87,"threads_started":1}
I20260812 16:37:08.568073 5166 heartbeater.cc:347] Connected to a master server at 127.0.253.254:41125
I20260812 16:37:08.568185 5166 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:08.568372 5166 heartbeater.cc:511] Master 127.0.253.254:41125 requested a full tablet report, sending...
I20260812 16:37:08.568770 4525 ts_manager.cc:194] Registered new tserver with Master: 7133565a63d24fe09d46b0f58def254b (127.0.253.194:37559)
I20260812 16:37:08.569242 4525 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.194@KRBTEST.COM'} at 127.0.253.194:40559
WARNING: no policy specified for kudu/127.0.253.195@KRBTEST.COM; defaulting to no policy
I20260812 16:37:08.575784 5183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:08.569204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41125 (local address 127.0.253.194:44027)
0812 16:37:08.569366 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:08.569379 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:08.569434 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:08.569574 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:08.569575 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:08.569841 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:08.569926 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:08.569928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.570609 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.570610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:08.571575 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:08.571579 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.571930 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.571931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:08.572031 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:08.572443 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:08.572450 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:08.572914 (+ 464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:08.574613 (+ 1699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:08.574614 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:08.574614 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:08.574801 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:08.575007 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:08.575008 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:08.575008 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:08.575118 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:08.575500 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:08.575501 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:08.575565 (+ 64us) client_negotiation.cc:769] Sending connection context
0812 16:37:08.575629 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 16:37:08.575713 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":82,"threads_started":1}
I20260812 16:37:08.575814 4642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:08.569433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:44027 (local address 127.0.253.254:41125)
0812 16:37:08.569468 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:08.569470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:08.569477 (+ 7us) server_negotiation.cc:407] Connection header received
0812 16:37:08.569496 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:08.569497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:08.569506 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:08.569562 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:08.570033 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.570505 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.571665 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.571848 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.571905 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:08.573021 (+ 1116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:08.573022 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:08.573023 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:08.573029 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:08.574524 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:08.574877 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:08.574878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:08.574878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:08.574938 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:08.575173 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:08.575174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:08.575175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:08.575400 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:08.575469 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:08.575684 (+ 215us) server_negotiation.cc:299] Negotiation successful
0812 16:37:08.575754 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab.
Entry for principal kudu/127.0.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab.
I20260812 16:37:08.594085 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.195:0
--local_ip_for_outbound_sockets=127.0.253.195
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:41125
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:33653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:08.701478 5192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:08.701685 5192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:08.701710 5192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:08.702773 5192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:08.702814 5192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:08.702836 5192 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 16:37:08.702847 5192 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 16:37:08.702864 5192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.195
I20260812 16:37:08.704344 5192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:33653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.253.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.253.254:41125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.195
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:08.704541 5192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:08.704726 5192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:08.710119 5205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:08.710110 5200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:08.710110 5203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:08.710367 5192 server_base.cc:1034] running on GCE node
I20260812 16:37:08.710667 5192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:08.710883 5192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:08.712023 5192 hybrid_clock.cc:648] HybridClock initialized: now 1786552628712004 us; error 34 us; skew 500 ppm
Aug 12 16:37:08 dist-test-slave-w4v5 krb5kdc[4447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552628, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:08.714443 5192 init.cc:377] Logged in from keytab as kudu/127.0.253.195@KRBTEST.COM (short username kudu)
I20260812 16:37:08.714833 5192 webserver.cc:460] Webserver started at http://127.0.253.195:39845/ using document root <none> and password file <none>
I20260812 16:37:08.715037 5192 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:08.715140 5192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:08.715246 5192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:08.716246 5192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-2/data/instance:
uuid: "e1f11bd30d53435680b94197726c74bf"
format_stamp: "Formatted at 2026-08-12 16:37:08 on dist-test-slave-w4v5"
server_key: "3d24ee07f2f94dd176f2481ef51f789e"
server_key_iv: "7738b9e0803c0d7e3eaa2c159576cb3a"
server_key_version: "encryptionkey@0"
I20260812 16:37:08.716535 5192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance:
uuid: "e1f11bd30d53435680b94197726c74bf"
format_stamp: "Formatted at 2026-08-12 16:37:08 on dist-test-slave-w4v5"
server_key: "3d24ee07f2f94dd176f2481ef51f789e"
server_key_iv: "7738b9e0803c0d7e3eaa2c159576cb3a"
server_key_version: "encryptionkey@0"
I20260812 16:37:08.717828 5192 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:37:08.718590 5216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:08.718806 5192 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:08.718915 5192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-2/wal
uuid: "e1f11bd30d53435680b94197726c74bf"
format_stamp: "Formatted at 2026-08-12 16:37:08 on dist-test-slave-w4v5"
server_key: "3d24ee07f2f94dd176f2481ef51f789e"
server_key_iv: "7738b9e0803c0d7e3eaa2c159576cb3a"
server_key_version: "encryptionkey@0"
I20260812 16:37:08.718993 5192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:08.734342 5192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:08.736506 5192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:08.736670 5192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:08.737267 5192 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:08.737327 5192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:08.737371 5192 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:08.737398 5192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 16:37:08 dist-test-slave-w4v5 krb5kdc[4447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552628, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:08.744956 5192 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.195:39525
I20260812 16:37:08.745043 5396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.195:39525 every 8 connection(s)
I20260812 16:37:08.745605 5192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
I20260812 16:37:08.748220 4642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:08.737696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:37221 (local address 127.0.253.254:41125)
0812 16:37:08.737754 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:08.737756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:08.737988 (+ 232us) server_negotiation.cc:407] Connection header received
0812 16:37:08.738559 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:08.738561 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:08.738575 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:08.738649 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:08.739672 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.740246 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.741346 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.741565 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.741642 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:08.745136 (+ 3494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:08.745138 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:08.745138 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:08.745310 (+ 172us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:08.746897 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:08.747349 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:08.747350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:08.747350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:08.747405 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:08.747620 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:08.747620 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:08.747620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:08.747723 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:08.747772 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:08.748083 (+ 311us) server_negotiation.cc:299] Negotiation successful
0812 16:37:08.748157 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 16:37:08.748420 5230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:08.737701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41125 (local address 127.0.253.195:37221)
0812 16:37:08.737860 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:08.737898 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:08.738436 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:08.738780 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:08.738785 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:08.739223 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:08.739543 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:08.739551 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.740353 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.740354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:08.741249 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:08.741252 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.741653 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.741654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:08.741843 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:08.742390 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:08.742402 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:08.745035 (+ 2633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:08.746975 (+ 1940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:08.746980 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:08.746982 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:08.747277 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:08.747467 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:08.747468 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:08.747468 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:08.747568 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:08.747802 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:08.747809 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:08.747936 (+ 127us) client_negotiation.cc:769] Sending connection context
0812 16:37:08.748042 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 16:37:08.748174 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":88,"threads_started":1}
I20260812 16:37:08.748981 5397 heartbeater.cc:347] Connected to a master server at 127.0.253.254:41125
I20260812 16:37:08.749069 5397 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:08.749266 5397 heartbeater.cc:511] Master 127.0.253.254:41125 requested a full tablet report, sending...
I20260812 16:37:08.749651 4525 ts_manager.cc:194] Registered new tserver with Master: e1f11bd30d53435680b94197726c74bf (127.0.253.195:39525)
I20260812 16:37:08.750142 4525 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.195@KRBTEST.COM'} at 127.0.253.195:37221
I20260812 16:37:08.752040 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 5192
I20260812 16:37:08.752141 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance
I20260812 16:37:08.752343 1015 external_mini_cluster.cc:1428] Setting key 170ec42dd8d367fb5cd86234df3552b4
I20260812 16:37:08.753193 1015 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:37:08.756095 4642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:08.750214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:48701 (local address 127.0.253.254:41125)
0812 16:37:08.750257 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:08.750259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:08.750335 (+ 76us) server_negotiation.cc:407] Connection header received
0812 16:37:08.750411 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:08.750412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:08.750422 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:08.750464 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:08.750857 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.751403 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.752202 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.752442 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.752504 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:08.753467 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:08.753468 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:08.753469 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:08.753476 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:08.754854 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:08.755307 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:08.755308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:08.755308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:08.755375 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:08.755616 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:08.755617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:08.755617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:08.755743 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:08.755810 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:08.755973 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 16:37:08.756035 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 16:37:08.756224 5403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:08.750145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41125 (local address 127.0.253.195:48701)
0812 16:37:08.750276 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:08.750287 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:08.750361 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:08.750478 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:08.750479 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:08.750704 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:08.750773 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:08.750775 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.751498 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.751499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:08.752097 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:08.752099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.752504 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.752505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:08.752589 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:08.753055 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:08.753060 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:08.753380 (+ 320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:08.754951 (+ 1571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:08.754952 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:08.754952 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:08.755221 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:08.755458 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:08.755459 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:08.755460 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:08.755525 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:08.755820 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:08.755822 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:08.755874 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 16:37:08.755957 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 16:37:08.756076 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":78,"threads_started":1}
Aug 12 16:37:08 dist-test-slave-w4v5 krb5kdc[4447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 16:37:08 dist-test-slave-w4v5 krb5kdc[4447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:08.781736 4642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:08.773740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38380 (local address 127.0.253.254:41125)
0812 16:37:08.773806 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:08.773817 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:08.773826 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:37:08.773852 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:08.773853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:08.773863 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:08.773915 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:08.774490 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.775004 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.775788 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.776062 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.776118 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:08.778533 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:08.778535 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:08.778535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:08.778542 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:08.780237 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:08.780703 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:08.780704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:08.780705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:08.780773 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:08.781106 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:08.781107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:08.781108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:08.781263 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:08.781390 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:08.781527 (+ 137us) server_negotiation.cc:299] Negotiation successful
0812 16:37:08.781617 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 16:37:08.783815 4525 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38380:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:37:08.785452 4525 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:37:08.792161 5438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:08.788271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51386 (local address 127.0.253.193:46443)
0812 16:37:08.788482 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:08.788485 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:08.788496 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:37:08.788554 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:08.788557 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:08.788572 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:08.788652 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:08.788899 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.789590 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.790860 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.791698 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.791785 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:08.791850 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:08.792022 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 16:37:08.792062 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":102,"threads_started":1}
I20260812 16:37:08.792155 5431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:08.787877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:46443 (local address 127.0.0.1:51386)
0812 16:37:08.788192 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:08.788203 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:08.788260 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:08.788701 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:08.788705 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:08.788708 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:08.788784 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:08.788787 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.789704 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.789705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:08.790786 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:08.790789 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.791784 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.791785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:08.791879 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:08.791893 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 16:37:08.792001 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 16:37:08.792057 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":108,"threads_started":1}
I20260812 16:37:08.792660 5430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:08.787876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:37559 (local address 127.0.0.1:59048)
0812 16:37:08.787996 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:08.788040 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:08.788104 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:08.788842 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:08.788843 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:08.788845 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:08.788907 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:08.788909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.790288 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.790289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:08.791463 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:08.791466 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.792432 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.792433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:08.792496 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:08.792500 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:37:08.792553 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 16:37:08.792593 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"mutex_wait_us":2,"thread_start_us":67,"threads_started":1}
I20260812 16:37:08.792742 5433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:08.787987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59048 (local address 127.0.253.194:37559)
0812 16:37:08.788471 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:08.788477 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:08.788490 (+ 13us) server_negotiation.cc:407] Connection header received
0812 16:37:08.788535 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:08.788537 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:08.788555 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:08.788990 (+ 435us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:08.789057 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.790174 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.791555 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.792364 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.792430 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:08.792481 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:08.792586 (+ 105us) server_negotiation.cc:299] Negotiation successful
0812 16:37:08.792638 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":94,"threads_started":1}
I20260812 16:37:08.793006 4790 tablet_service.cc:1467] Processing CreateTablet for tablet fc79fb036cec47f5b949b37573764fbb (DEFAULT_TABLE table=test-table [id=488af6b45b55462982695c5698cbc178]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:08.793327 5434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:08.788102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.195:39525 (local address 127.0.0.1:41812)
0812 16:37:08.788684 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:08.788695 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:08.788748 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:08.788984 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:08.788985 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:08.788987 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:08.789017 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:08.789018 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.790266 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.790268 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:08.791297 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:08.791300 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.792567 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.792568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:08.792627 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:08.792632 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:37:08.792678 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 16:37:08.793263 (+ 585us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"mutex_wait_us":24,"thread_start_us":79,"threads_started":1}
I20260812 16:37:08.793172 5436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:08.788220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41812 (local address 127.0.253.195:39525)
0812 16:37:08.788422 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:08.788427 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:08.788744 (+ 317us) server_negotiation.cc:407] Connection header received
0812 16:37:08.788818 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:08.788820 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:08.788837 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:08.788911 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:08.789092 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.790167 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.791391 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.792099 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.792183 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:08.792991 (+ 808us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:08.793021 (+ 30us) server_negotiation.cc:299] Negotiation successful
0812 16:37:08.793066 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":97,"threads_started":1}
I20260812 16:37:08.793443 4790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc79fb036cec47f5b949b37573764fbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:08.793434 5039 tablet_service.cc:1467] Processing CreateTablet for tablet fc79fb036cec47f5b949b37573764fbb (DEFAULT_TABLE table=test-table [id=488af6b45b55462982695c5698cbc178]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:08.793793 5039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc79fb036cec47f5b949b37573764fbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:08.795817 5296 tablet_service.cc:1467] Processing CreateTablet for tablet fc79fb036cec47f5b949b37573764fbb (DEFAULT_TABLE table=test-table [id=488af6b45b55462982695c5698cbc178]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:08.796205 5296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc79fb036cec47f5b949b37573764fbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:08.796947 5445 tablet_bootstrap.cc:492] T fc79fb036cec47f5b949b37573764fbb P b36bcb39624f44849c3b9ccfadfbf34d: Bootstrap starting.
I20260812 16:37:08.797165 5439 tablet_bootstrap.cc:492] T fc79fb036cec47f5b949b37573764fbb P 7133565a63d24fe09d46b0f58def254b: Bootstrap starting.
I20260812 16:37:08.797729 5439 tablet_bootstrap.cc:654] T fc79fb036cec47f5b949b37573764fbb P 7133565a63d24fe09d46b0f58def254b: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:08.798040 5439 log.cc:824] T fc79fb036cec47f5b949b37573764fbb P 7133565a63d24fe09d46b0f58def254b: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:08.798832 5445 tablet_bootstrap.cc:654] T fc79fb036cec47f5b949b37573764fbb P b36bcb39624f44849c3b9ccfadfbf34d: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:08.798905 5439 tablet_bootstrap.cc:492] T fc79fb036cec47f5b949b37573764fbb P 7133565a63d24fe09d46b0f58def254b: No bootstrap required, opened a new log
I20260812 16:37:08.798969 5439 ts_tablet_manager.cc:1397] T fc79fb036cec47f5b949b37573764fbb P 7133565a63d24fe09d46b0f58def254b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:37:08.799366 5445 log.cc:824] T fc79fb036cec47f5b949b37573764fbb P b36bcb39624f44849c3b9ccfadfbf34d: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:08.799559 5448 tablet_bootstrap.cc:492] T fc79fb036cec47f5b949b37573764fbb P e1f11bd30d53435680b94197726c74bf: Bootstrap starting.
I20260812 16:37:08.800181 5445 tablet_bootstrap.cc:492] T fc79fb036cec47f5b949b37573764fbb P b36bcb39624f44849c3b9ccfadfbf34d: No bootstrap required, opened a new log
I20260812 16:37:08.800257 5445 ts_tablet_manager.cc:1397] T fc79fb036cec47f5b949b37573764fbb P b36bcb39624f44849c3b9ccfadfbf34d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 16:37:08.800410 5448 tablet_bootstrap.cc:654] T fc79fb036cec47f5b949b37573764fbb P e1f11bd30d53435680b94197726c74bf: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:08.800729 5448 log.cc:824] T fc79fb036cec47f5b949b37573764fbb P e1f11bd30d53435680b94197726c74bf: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:08.801149 5439 raft_consensus.cc:348] T fc79fb036cec47f5b949b37573764fbb P 7133565a63d24fe09d46b0f58def254b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:08.801324 5439 raft_consensus.cc:374] T fc79fb036cec47f5b949b37573764fbb P 7133565a63d24fe09d46b0f58def254b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:08.801362 5439 raft_consensus.cc:729] T fc79fb036cec47f5b949b37573764fbb P 7133565a63d24fe09d46b0f58def254b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7133565a63d24fe09d46b0f58def254b, State: Initialized, Role: FOLLOWER
I20260812 16:37:08.801473 5448 tablet_bootstrap.cc:492] T fc79fb036cec47f5b949b37573764fbb P e1f11bd30d53435680b94197726c74bf: No bootstrap required, opened a new log
I20260812 16:37:08.801542 5448 ts_tablet_manager.cc:1397] T fc79fb036cec47f5b949b37573764fbb P e1f11bd30d53435680b94197726c74bf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:37:08.801501 5439 consensus_queue.cc:260] T fc79fb036cec47f5b949b37573764fbb P 7133565a63d24fe09d46b0f58def254b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:08.801783 5439 ts_tablet_manager.cc:1428] T fc79fb036cec47f5b949b37573764fbb P 7133565a63d24fe09d46b0f58def254b: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 16:37:08.801818 5166 heartbeater.cc:503] Master 127.0.253.254:41125 was elected leader, sending a full tablet report...
I20260812 16:37:08.802336 5445 raft_consensus.cc:348] T fc79fb036cec47f5b949b37573764fbb P b36bcb39624f44849c3b9ccfadfbf34d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:08.802498 5445 raft_consensus.cc:374] T fc79fb036cec47f5b949b37573764fbb P b36bcb39624f44849c3b9ccfadfbf34d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:08.802542 5445 raft_consensus.cc:729] T fc79fb036cec47f5b949b37573764fbb P b36bcb39624f44849c3b9ccfadfbf34d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b36bcb39624f44849c3b9ccfadfbf34d, State: Initialized, Role: FOLLOWER
I20260812 16:37:08.802695 5445 consensus_queue.cc:260] T fc79fb036cec47f5b949b37573764fbb P b36bcb39624f44849c3b9ccfadfbf34d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:08.803004 4917 heartbeater.cc:503] Master 127.0.253.254:41125 was elected leader, sending a full tablet report...
I20260812 16:37:08.803293 5445 ts_tablet_manager.cc:1428] T fc79fb036cec47f5b949b37573764fbb P b36bcb39624f44849c3b9ccfadfbf34d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:37:08.803949 5448 raft_consensus.cc:348] T fc79fb036cec47f5b949b37573764fbb P e1f11bd30d53435680b94197726c74bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:08.804126 5448 raft_consensus.cc:374] T fc79fb036cec47f5b949b37573764fbb P e1f11bd30d53435680b94197726c74bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:08.804173 5448 raft_consensus.cc:729] T fc79fb036cec47f5b949b37573764fbb P e1f11bd30d53435680b94197726c74bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f11bd30d53435680b94197726c74bf, State: Initialized, Role: FOLLOWER
I20260812 16:37:08.804286 5448 consensus_queue.cc:260] T fc79fb036cec47f5b949b37573764fbb P e1f11bd30d53435680b94197726c74bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:08.804533 5448 ts_tablet_manager.cc:1428] T fc79fb036cec47f5b949b37573764fbb P e1f11bd30d53435680b94197726c74bf: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 16:37:08.804607 5397 heartbeater.cc:503] Master 127.0.253.254:41125 was elected leader, sending a full tablet report...
W20260812 16:37:08.811898 5170 tablet.cc:2369] T fc79fb036cec47f5b949b37573764fbb P 7133565a63d24fe09d46b0f58def254b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:37:08.848968 5472 raft_consensus.cc:482] T fc79fb036cec47f5b949b37573764fbb P e1f11bd30d53435680b94197726c74bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:37:08.849177 5472 raft_consensus.cc:504] T fc79fb036cec47f5b949b37573764fbb P e1f11bd30d53435680b94197726c74bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:08.849515 5472 leader_election.cc:288] T fc79fb036cec47f5b949b37573764fbb P e1f11bd30d53435680b94197726c74bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b36bcb39624f44849c3b9ccfadfbf34d (127.0.253.193:46443), 7133565a63d24fe09d46b0f58def254b (127.0.253.194:37559)
I20260812 16:37:08.852751 5433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:08.849893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:38363 (local address 127.0.253.194:37559)
0812 16:37:08.849936 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:08.849938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:08.849946 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:37:08.849979 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:08.849980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:08.849991 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:08.850070 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:08.850307 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.850846 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.851935 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.852384 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.852469 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:08.852479 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:08.852632 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 16:37:08.852676 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 16:37:08.852780 5230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:08.849771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:37559 (local address 127.0.253.195:38363)
0812 16:37:08.849818 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:08.849844 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:08.849892 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:08.850130 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:08.850132 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:08.850134 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:08.850219 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:08.850221 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.850940 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.850941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:08.851845 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:08.851848 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.852466 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.852467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:08.852555 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:08.852558 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:37:08.852624 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 16:37:08.852665 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260812 16:37:08.852784 5438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:08.849833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:47073 (local address 127.0.253.193:46443)
0812 16:37:08.849882 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:08.849885 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:08.850108 (+ 223us) server_negotiation.cc:407] Connection header received
0812 16:37:08.850141 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:08.850142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:08.850153 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:08.850230 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:08.850449 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.851010 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.851980 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.852400 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.852486 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:08.852495 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:08.852672 (+ 177us) server_negotiation.cc:299] Negotiation successful
0812 16:37:08.852715 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 16:37:08.852805 5475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:08.849774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:46443 (local address 127.0.253.195:47073)
0812 16:37:08.850000 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:08.850010 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:08.850074 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:08.850238 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:08.850239 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:08.850250 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:08.850347 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:08.850348 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.851133 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.851134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:08.851867 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:08.851870 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.852486 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.852488 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:08.852573 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:08.852577 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:37:08.852657 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 16:37:08.852698 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":50,"thread_start_us":124,"threads_started":1}
I20260812 16:37:08.853200 4824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc79fb036cec47f5b949b37573764fbb" candidate_uuid: "e1f11bd30d53435680b94197726c74bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" is_pre_election: true
I20260812 16:37:08.853410 4824 raft_consensus.cc:2454] T fc79fb036cec47f5b949b37573764fbb P b36bcb39624f44849c3b9ccfadfbf34d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1f11bd30d53435680b94197726c74bf in term 0.
I20260812 16:37:08.853381 5075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc79fb036cec47f5b949b37573764fbb" candidate_uuid: "e1f11bd30d53435680b94197726c74bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7133565a63d24fe09d46b0f58def254b" is_pre_election: true
I20260812 16:37:08.853547 5075 raft_consensus.cc:2454] T fc79fb036cec47f5b949b37573764fbb P 7133565a63d24fe09d46b0f58def254b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1f11bd30d53435680b94197726c74bf in term 0.
I20260812 16:37:08.853708 5220 leader_election.cc:302] T fc79fb036cec47f5b949b37573764fbb P e1f11bd30d53435680b94197726c74bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b36bcb39624f44849c3b9ccfadfbf34d, e1f11bd30d53435680b94197726c74bf; no voters:
I20260812 16:37:08.853888 5472 raft_consensus.cc:2781] T fc79fb036cec47f5b949b37573764fbb P e1f11bd30d53435680b94197726c74bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:37:08.853956 5472 raft_consensus.cc:482] T fc79fb036cec47f5b949b37573764fbb P e1f11bd30d53435680b94197726c74bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:37:08.853974 5472 raft_consensus.cc:3037] T fc79fb036cec47f5b949b37573764fbb P e1f11bd30d53435680b94197726c74bf [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:08.854655 5472 raft_consensus.cc:504] T fc79fb036cec47f5b949b37573764fbb P e1f11bd30d53435680b94197726c74bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:08.854802 5472 leader_election.cc:288] T fc79fb036cec47f5b949b37573764fbb P e1f11bd30d53435680b94197726c74bf [CANDIDATE]: Term 1 election: Requested vote from peers b36bcb39624f44849c3b9ccfadfbf34d (127.0.253.193:46443), 7133565a63d24fe09d46b0f58def254b (127.0.253.194:37559)
I20260812 16:37:08.854961 4824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc79fb036cec47f5b949b37573764fbb" candidate_uuid: "e1f11bd30d53435680b94197726c74bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b36bcb39624f44849c3b9ccfadfbf34d"
I20260812 16:37:08.854974 5075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc79fb036cec47f5b949b37573764fbb" candidate_uuid: "e1f11bd30d53435680b94197726c74bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7133565a63d24fe09d46b0f58def254b"
I20260812 16:37:08.855048 4824 raft_consensus.cc:3037] T fc79fb036cec47f5b949b37573764fbb P b36bcb39624f44849c3b9ccfadfbf34d [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:08.855048 5075 raft_consensus.cc:3037] T fc79fb036cec47f5b949b37573764fbb P 7133565a63d24fe09d46b0f58def254b [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:08.855970 5075 raft_consensus.cc:2454] T fc79fb036cec47f5b949b37573764fbb P 7133565a63d24fe09d46b0f58def254b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1f11bd30d53435680b94197726c74bf in term 1.
I20260812 16:37:08.855970 4824 raft_consensus.cc:2454] T fc79fb036cec47f5b949b37573764fbb P b36bcb39624f44849c3b9ccfadfbf34d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1f11bd30d53435680b94197726c74bf in term 1.
I20260812 16:37:08.856153 5223 leader_election.cc:302] T fc79fb036cec47f5b949b37573764fbb P e1f11bd30d53435680b94197726c74bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7133565a63d24fe09d46b0f58def254b, e1f11bd30d53435680b94197726c74bf; no voters:
I20260812 16:37:08.856264 5472 raft_consensus.cc:2781] T fc79fb036cec47f5b949b37573764fbb P e1f11bd30d53435680b94197726c74bf [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:37:08.856423 5472 raft_consensus.cc:686] T fc79fb036cec47f5b949b37573764fbb P e1f11bd30d53435680b94197726c74bf [term 1 LEADER]: Becoming Leader. State: Replica: e1f11bd30d53435680b94197726c74bf, State: Running, Role: LEADER
I20260812 16:37:08.856530 5472 consensus_queue.cc:237] T fc79fb036cec47f5b949b37573764fbb P e1f11bd30d53435680b94197726c74bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:08.857415 4525 catalog_manager.cc:5495] T fc79fb036cec47f5b949b37573764fbb P e1f11bd30d53435680b94197726c74bf reported cstate change: term changed from 0 to 1, leader changed from <none> to e1f11bd30d53435680b94197726c74bf (127.0.253.195). New cstate: current_term: 1 leader_uuid: "e1f11bd30d53435680b94197726c74bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } health_report { overall_health: HEALTHY } } }
W20260812 16:37:08.882577 4918 tablet.cc:2369] T fc79fb036cec47f5b949b37573764fbb P b36bcb39624f44849c3b9ccfadfbf34d: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 16:37:08 dist-test-slave-w4v5 krb5kdc[4447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552628, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:08.890900 5485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:08.883833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41125 (local address 127.0.0.1:38382)
0812 16:37:08.883973 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:08.883983 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:08.884048 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:08.884191 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:08.884192 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:08.884562 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:08.884615 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:08.884618 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.885348 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.885349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:08.885904 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:08.885905 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.886270 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.886271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:08.886316 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:08.886893 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:08.886898 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:08.888464 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:08.889950 (+ 1486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:08.889958 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:08.889960 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:08.890153 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:08.890328 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:08.890329 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:08.890329 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:08.890388 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:08.890600 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:08.890606 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:08.890652 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 16:37:08.890705 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 16:37:08.890805 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":62,"threads_started":1}
I20260812 16:37:08.890996 4642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:08.883908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38382 (local address 127.0.253.254:41125)
0812 16:37:08.883997 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:08.883999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:08.884049 (+ 50us) server_negotiation.cc:407] Connection header received
0812 16:37:08.884118 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:08.884119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:08.884128 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:08.884170 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:08.884781 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.885280 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.885988 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.886199 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.886255 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:08.888566 (+ 2311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:08.888568 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:08.888568 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:08.888575 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:08.889876 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:08.890220 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:08.890220 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:08.890221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:08.890270 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:08.890441 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:08.890441 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:08.890441 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:08.890548 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:08.890597 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:08.890739 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 16:37:08.890853 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":57}
I20260812 16:37:08.893158 4525 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:38382:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 16:37:08.893445 4525 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:37:08.895331 5039 tablet_service.cc:1467] Processing CreateTablet for tablet e909db59736a433eb8ede81ceec67aff (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f197e556e0fc4e5089e2536dfbcb3ad7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 16:37:08.895545 5039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e909db59736a433eb8ede81ceec67aff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:08.895481 5296 tablet_service.cc:1467] Processing CreateTablet for tablet e909db59736a433eb8ede81ceec67aff (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f197e556e0fc4e5089e2536dfbcb3ad7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 16:37:08.895690 5296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e909db59736a433eb8ede81ceec67aff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:08.897045 5439 tablet_bootstrap.cc:492] T e909db59736a433eb8ede81ceec67aff P 7133565a63d24fe09d46b0f58def254b: Bootstrap starting.
I20260812 16:37:08.897122 5448 tablet_bootstrap.cc:492] T e909db59736a433eb8ede81ceec67aff P e1f11bd30d53435680b94197726c74bf: Bootstrap starting.
I20260812 16:37:08.897244 4790 tablet_service.cc:1467] Processing CreateTablet for tablet e909db59736a433eb8ede81ceec67aff (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f197e556e0fc4e5089e2536dfbcb3ad7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 16:37:08.897375 4790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e909db59736a433eb8ede81ceec67aff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:08.897675 5448 tablet_bootstrap.cc:654] T e909db59736a433eb8ede81ceec67aff P e1f11bd30d53435680b94197726c74bf: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:08.897845 5439 tablet_bootstrap.cc:654] T e909db59736a433eb8ede81ceec67aff P 7133565a63d24fe09d46b0f58def254b: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:08.898515 5448 tablet_bootstrap.cc:492] T e909db59736a433eb8ede81ceec67aff P e1f11bd30d53435680b94197726c74bf: No bootstrap required, opened a new log
I20260812 16:37:08.898561 5445 tablet_bootstrap.cc:492] T e909db59736a433eb8ede81ceec67aff P b36bcb39624f44849c3b9ccfadfbf34d: Bootstrap starting.
I20260812 16:37:08.898576 5448 ts_tablet_manager.cc:1397] T e909db59736a433eb8ede81ceec67aff P e1f11bd30d53435680b94197726c74bf: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 16:37:08.898622 5439 tablet_bootstrap.cc:492] T e909db59736a433eb8ede81ceec67aff P 7133565a63d24fe09d46b0f58def254b: No bootstrap required, opened a new log
I20260812 16:37:08.898663 5439 ts_tablet_manager.cc:1397] T e909db59736a433eb8ede81ceec67aff P 7133565a63d24fe09d46b0f58def254b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 16:37:08.898723 5448 raft_consensus.cc:348] T e909db59736a433eb8ede81ceec67aff P e1f11bd30d53435680b94197726c74bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:08.898772 5448 raft_consensus.cc:374] T e909db59736a433eb8ede81ceec67aff P e1f11bd30d53435680b94197726c74bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:08.898787 5448 raft_consensus.cc:729] T e909db59736a433eb8ede81ceec67aff P e1f11bd30d53435680b94197726c74bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f11bd30d53435680b94197726c74bf, State: Initialized, Role: FOLLOWER
I20260812 16:37:08.898847 5448 consensus_queue.cc:260] T e909db59736a433eb8ede81ceec67aff P e1f11bd30d53435680b94197726c74bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:08.898839 5439 raft_consensus.cc:348] T e909db59736a433eb8ede81ceec67aff P 7133565a63d24fe09d46b0f58def254b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:08.898936 5439 raft_consensus.cc:374] T e909db59736a433eb8ede81ceec67aff P 7133565a63d24fe09d46b0f58def254b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:08.898942 5448 ts_tablet_manager.cc:1428] T e909db59736a433eb8ede81ceec67aff P e1f11bd30d53435680b94197726c74bf: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:08.898967 5439 raft_consensus.cc:729] T e909db59736a433eb8ede81ceec67aff P 7133565a63d24fe09d46b0f58def254b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7133565a63d24fe09d46b0f58def254b, State: Initialized, Role: FOLLOWER
I20260812 16:37:08.898988 5445 tablet_bootstrap.cc:654] T e909db59736a433eb8ede81ceec67aff P b36bcb39624f44849c3b9ccfadfbf34d: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:08.898979 5472 tablet_replica.cc:439] T e909db59736a433eb8ede81ceec67aff P e1f11bd30d53435680b94197726c74bf: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } } }
I20260812 16:37:08.899048 5472 tablet_replica.cc:442] T e909db59736a433eb8ede81ceec67aff P e1f11bd30d53435680b94197726c74bf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:37:08.899034 5439 consensus_queue.cc:260] T e909db59736a433eb8ede81ceec67aff P 7133565a63d24fe09d46b0f58def254b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:08.899188 5439 ts_tablet_manager.cc:1428] T e909db59736a433eb8ede81ceec67aff P 7133565a63d24fe09d46b0f58def254b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 16:37:08.899266 5470 tablet_replica.cc:439] T e909db59736a433eb8ede81ceec67aff P 7133565a63d24fe09d46b0f58def254b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } } }
I20260812 16:37:08.899360 5470 tablet_replica.cc:442] T e909db59736a433eb8ede81ceec67aff P 7133565a63d24fe09d46b0f58def254b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:37:08.899947 5445 tablet_bootstrap.cc:492] T e909db59736a433eb8ede81ceec67aff P b36bcb39624f44849c3b9ccfadfbf34d: No bootstrap required, opened a new log
I20260812 16:37:08.899992 5445 ts_tablet_manager.cc:1397] T e909db59736a433eb8ede81ceec67aff P b36bcb39624f44849c3b9ccfadfbf34d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:08.900122 5445 raft_consensus.cc:348] T e909db59736a433eb8ede81ceec67aff P b36bcb39624f44849c3b9ccfadfbf34d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:08.900175 5445 raft_consensus.cc:374] T e909db59736a433eb8ede81ceec67aff P b36bcb39624f44849c3b9ccfadfbf34d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:08.900189 5445 raft_consensus.cc:729] T e909db59736a433eb8ede81ceec67aff P b36bcb39624f44849c3b9ccfadfbf34d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b36bcb39624f44849c3b9ccfadfbf34d, State: Initialized, Role: FOLLOWER
I20260812 16:37:08.900250 5445 consensus_queue.cc:260] T e909db59736a433eb8ede81ceec67aff P b36bcb39624f44849c3b9ccfadfbf34d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:08.900377 5445 ts_tablet_manager.cc:1428] T e909db59736a433eb8ede81ceec67aff P b36bcb39624f44849c3b9ccfadfbf34d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 16:37:08.900413 5471 tablet_replica.cc:439] T e909db59736a433eb8ede81ceec67aff P b36bcb39624f44849c3b9ccfadfbf34d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } } }
I20260812 16:37:08.900501 5471 tablet_replica.cc:442] T e909db59736a433eb8ede81ceec67aff P b36bcb39624f44849c3b9ccfadfbf34d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:37:08.960551 5470 raft_consensus.cc:482] T e909db59736a433eb8ede81ceec67aff P 7133565a63d24fe09d46b0f58def254b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:37:08.960652 5470 raft_consensus.cc:504] T e909db59736a433eb8ede81ceec67aff P 7133565a63d24fe09d46b0f58def254b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:08.961027 5470 leader_election.cc:288] T e909db59736a433eb8ede81ceec67aff P 7133565a63d24fe09d46b0f58def254b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b36bcb39624f44849c3b9ccfadfbf34d (127.0.253.193:46443), e1f11bd30d53435680b94197726c74bf (127.0.253.195:39525)
I20260812 16:37:08.964208 5438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:08.961305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:55421 (local address 127.0.253.193:46443)
0812 16:37:08.961400 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:08.961416 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:08.961428 (+ 12us) server_negotiation.cc:407] Connection header received
0812 16:37:08.961461 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:08.961462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:08.961473 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:08.961583 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:08.961761 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.962304 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.963444 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.963816 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.963908 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:08.963918 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:08.964084 (+ 166us) server_negotiation.cc:299] Negotiation successful
0812 16:37:08.964126 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 16:37:08.964232 5436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:08.961464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:58745 (local address 127.0.253.195:39525)
0812 16:37:08.961547 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:08.961549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:08.961558 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:37:08.961610 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:08.961610 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:08.961620 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:08.961677 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:08.961839 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.962301 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.963458 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.963841 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.963940 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:08.963951 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:08.964106 (+ 155us) server_negotiation.cc:299] Negotiation successful
0812 16:37:08.964148 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 16:37:08.964269 4968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:08.961223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:46443 (local address 127.0.253.194:55421)
0812 16:37:08.961294 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:08.961307 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:08.961358 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:08.961590 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:08.961592 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:08.961594 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:08.961676 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:08.961678 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.962397 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.962398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:08.963351 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:08.963354 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.963904 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.963905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:08.964002 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:08.964005 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:37:08.964074 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 16:37:08.964122 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":20}
I20260812 16:37:08.964293 5493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:08.961334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.195:39525 (local address 127.0.253.194:58745)
0812 16:37:08.961492 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:08.961503 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:08.961547 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:08.961684 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:08.961685 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:08.961687 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:08.961767 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:08.961769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.962390 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.962391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:08.963366 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:08.963369 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.963964 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.963965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:08.964027 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:08.964030 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:37:08.964099 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 16:37:08.964153 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":33,"thread_start_us":108,"threads_started":1}
I20260812 16:37:08.964581 4824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e909db59736a433eb8ede81ceec67aff" candidate_uuid: "7133565a63d24fe09d46b0f58def254b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" is_pre_election: true
I20260812 16:37:08.964695 4824 raft_consensus.cc:2454] T e909db59736a433eb8ede81ceec67aff P b36bcb39624f44849c3b9ccfadfbf34d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7133565a63d24fe09d46b0f58def254b in term 0.
I20260812 16:37:08.964658 5324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e909db59736a433eb8ede81ceec67aff" candidate_uuid: "7133565a63d24fe09d46b0f58def254b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f11bd30d53435680b94197726c74bf" is_pre_election: true
I20260812 16:37:08.964818 5324 raft_consensus.cc:2454] T e909db59736a433eb8ede81ceec67aff P e1f11bd30d53435680b94197726c74bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7133565a63d24fe09d46b0f58def254b in term 0.
I20260812 16:37:08.964910 4961 leader_election.cc:302] T e909db59736a433eb8ede81ceec67aff P 7133565a63d24fe09d46b0f58def254b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7133565a63d24fe09d46b0f58def254b, b36bcb39624f44849c3b9ccfadfbf34d; no voters:
I20260812 16:37:08.965042 5470 raft_consensus.cc:2781] T e909db59736a433eb8ede81ceec67aff P 7133565a63d24fe09d46b0f58def254b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:37:08.965114 5470 raft_consensus.cc:482] T e909db59736a433eb8ede81ceec67aff P 7133565a63d24fe09d46b0f58def254b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:37:08.965161 5470 raft_consensus.cc:3037] T e909db59736a433eb8ede81ceec67aff P 7133565a63d24fe09d46b0f58def254b [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:08.965885 5470 raft_consensus.cc:504] T e909db59736a433eb8ede81ceec67aff P 7133565a63d24fe09d46b0f58def254b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:08.966027 5470 leader_election.cc:288] T e909db59736a433eb8ede81ceec67aff P 7133565a63d24fe09d46b0f58def254b [CANDIDATE]: Term 1 election: Requested vote from peers b36bcb39624f44849c3b9ccfadfbf34d (127.0.253.193:46443), e1f11bd30d53435680b94197726c74bf (127.0.253.195:39525)
I20260812 16:37:08.966259 5324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e909db59736a433eb8ede81ceec67aff" candidate_uuid: "7133565a63d24fe09d46b0f58def254b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f11bd30d53435680b94197726c74bf"
I20260812 16:37:08.966249 4824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e909db59736a433eb8ede81ceec67aff" candidate_uuid: "7133565a63d24fe09d46b0f58def254b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b36bcb39624f44849c3b9ccfadfbf34d"
I20260812 16:37:08.966331 5324 raft_consensus.cc:3037] T e909db59736a433eb8ede81ceec67aff P e1f11bd30d53435680b94197726c74bf [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:08.966331 4824 raft_consensus.cc:3037] T e909db59736a433eb8ede81ceec67aff P b36bcb39624f44849c3b9ccfadfbf34d [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:08.967178 4824 raft_consensus.cc:2454] T e909db59736a433eb8ede81ceec67aff P b36bcb39624f44849c3b9ccfadfbf34d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7133565a63d24fe09d46b0f58def254b in term 1.
I20260812 16:37:08.967198 5324 raft_consensus.cc:2454] T e909db59736a433eb8ede81ceec67aff P e1f11bd30d53435680b94197726c74bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7133565a63d24fe09d46b0f58def254b in term 1.
I20260812 16:37:08.967355 4961 leader_election.cc:302] T e909db59736a433eb8ede81ceec67aff P 7133565a63d24fe09d46b0f58def254b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7133565a63d24fe09d46b0f58def254b, b36bcb39624f44849c3b9ccfadfbf34d; no voters:
I20260812 16:37:08.967495 5470 raft_consensus.cc:2781] T e909db59736a433eb8ede81ceec67aff P 7133565a63d24fe09d46b0f58def254b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:37:08.967674 5470 raft_consensus.cc:686] T e909db59736a433eb8ede81ceec67aff P 7133565a63d24fe09d46b0f58def254b [term 1 LEADER]: Becoming Leader. State: Replica: 7133565a63d24fe09d46b0f58def254b, State: Running, Role: LEADER
I20260812 16:37:08.967789 5470 consensus_queue.cc:237] T e909db59736a433eb8ede81ceec67aff P 7133565a63d24fe09d46b0f58def254b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:08.968297 5494 tablet_replica.cc:439] T e909db59736a433eb8ede81ceec67aff P 7133565a63d24fe09d46b0f58def254b: TxnStatusTablet state changed. Reason: New leader 7133565a63d24fe09d46b0f58def254b. Latest consensus state: current_term: 1 leader_uuid: "7133565a63d24fe09d46b0f58def254b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } } }
I20260812 16:37:08.968400 5494 tablet_replica.cc:442] T e909db59736a433eb8ede81ceec67aff P 7133565a63d24fe09d46b0f58def254b: This TxnStatusTablet replica's current role is: LEADER
I20260812 16:37:08.968624 4526 catalog_manager.cc:5495] T e909db59736a433eb8ede81ceec67aff P 7133565a63d24fe09d46b0f58def254b reported cstate change: term changed from 0 to 1, leader changed from <none> to 7133565a63d24fe09d46b0f58def254b (127.0.253.194). New cstate: current_term: 1 leader_uuid: "7133565a63d24fe09d46b0f58def254b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } health_report { overall_health: UNKNOWN } } }
I20260812 16:37:08.968979 5498 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 16:37:08.969106 5498 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 16:37:08.993227 5485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:08.990382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:37559 (local address 127.0.0.1:59062)
0812 16:37:08.990459 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:08.990472 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:08.990537 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:08.990751 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:08.990752 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:08.990754 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 16:37:08.990808 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:08.990811 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.991558 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.991559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:08.992179 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:08.992181 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.992660 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.992661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:08.992740 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:08.992755 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 16:37:08.993050 (+ 295us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 16:37:08.993061 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 16:37:08.993103 (+ 42us) client_negotiation.cc:240] Negotiation successful
0812 16:37:08.993155 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 16:37:08.993242 5433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:08.990461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59062 (local address 127.0.253.194:37559)
0812 16:37:08.990529 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:08.990531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:08.990542 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:37:08.990626 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:08.990629 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:08.990656 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:08.990744 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:37:08.990892 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.991470 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.992365 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:08.992578 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:08.992658 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:08.992861 (+ 203us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:37:08.992970 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:37:08.993069 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:08.993118 (+ 49us) server_negotiation.cc:299] Negotiation successful
0812 16:37:08.993140 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 16:37:08.995394 4824 raft_consensus.cc:1261] T e909db59736a433eb8ede81ceec67aff P b36bcb39624f44849c3b9ccfadfbf34d [term 1 FOLLOWER]: Refusing update from remote peer 7133565a63d24fe09d46b0f58def254b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:37:08.995404 5324 raft_consensus.cc:1261] T e909db59736a433eb8ede81ceec67aff P e1f11bd30d53435680b94197726c74bf [term 1 FOLLOWER]: Refusing update from remote peer 7133565a63d24fe09d46b0f58def254b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:37:08.995786 5470 consensus_queue.cc:1035] T e909db59736a433eb8ede81ceec67aff P 7133565a63d24fe09d46b0f58def254b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:37:08.995929 5494 consensus_queue.cc:1035] T e909db59736a433eb8ede81ceec67aff P 7133565a63d24fe09d46b0f58def254b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 16:37:08.996345 5398 tablet.cc:2369] T e909db59736a433eb8ede81ceec67aff P e1f11bd30d53435680b94197726c74bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:37:08.997120 5471 tablet_replica.cc:439] T e909db59736a433eb8ede81ceec67aff P b36bcb39624f44849c3b9ccfadfbf34d: TxnStatusTablet state changed. Reason: New leader 7133565a63d24fe09d46b0f58def254b. Latest consensus state: current_term: 1 leader_uuid: "7133565a63d24fe09d46b0f58def254b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } } }
I20260812 16:37:08.997206 5471 tablet_replica.cc:442] T e909db59736a433eb8ede81ceec67aff P b36bcb39624f44849c3b9ccfadfbf34d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:37:08.997215 5472 tablet_replica.cc:439] T e909db59736a433eb8ede81ceec67aff P e1f11bd30d53435680b94197726c74bf: TxnStatusTablet state changed. Reason: New leader 7133565a63d24fe09d46b0f58def254b. Latest consensus state: current_term: 1 leader_uuid: "7133565a63d24fe09d46b0f58def254b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } } }
I20260812 16:37:08.997298 5472 tablet_replica.cc:442] T e909db59736a433eb8ede81ceec67aff P e1f11bd30d53435680b94197726c74bf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:37:08.998252 5494 tablet_replica.cc:439] T e909db59736a433eb8ede81ceec67aff P 7133565a63d24fe09d46b0f58def254b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7133565a63d24fe09d46b0f58def254b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } } }
I20260812 16:37:08.998344 5494 tablet_replica.cc:442] T e909db59736a433eb8ede81ceec67aff P 7133565a63d24fe09d46b0f58def254b: This TxnStatusTablet replica's current role is: LEADER
I20260812 16:37:08.998518 5472 tablet_replica.cc:439] T e909db59736a433eb8ede81ceec67aff P e1f11bd30d53435680b94197726c74bf: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7133565a63d24fe09d46b0f58def254b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } } }
I20260812 16:37:08.998600 5472 tablet_replica.cc:442] T e909db59736a433eb8ede81ceec67aff P e1f11bd30d53435680b94197726c74bf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:37:08.998848 5506 mvcc.cc:205] Tried to move back new op lower bound from 7317719568362053632 to 7317719568252624896. Current Snapshot: MvccSnapshot[applied={T|T < 7317719568362053632}]
I20260812 16:37:08.999894 5494 tablet_replica.cc:439] T e909db59736a433eb8ede81ceec67aff P 7133565a63d24fe09d46b0f58def254b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7133565a63d24fe09d46b0f58def254b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } } }
I20260812 16:37:08.999989 5494 tablet_replica.cc:442] T e909db59736a433eb8ede81ceec67aff P 7133565a63d24fe09d46b0f58def254b: This TxnStatusTablet replica's current role is: LEADER
I20260812 16:37:09.000525 5471 tablet_replica.cc:439] T e909db59736a433eb8ede81ceec67aff P b36bcb39624f44849c3b9ccfadfbf34d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7133565a63d24fe09d46b0f58def254b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } } }
I20260812 16:37:09.000715 5471 tablet_replica.cc:442] T e909db59736a433eb8ede81ceec67aff P b36bcb39624f44849c3b9ccfadfbf34d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:37:09.000919 5504 mvcc.cc:205] Tried to move back new op lower bound from 7317719568362053632 to 7317719568252624896. Current Snapshot: MvccSnapshot[applied={T|T < 7317719568362053632}]
I20260812 16:37:09.003607 5436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:09.000424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41822 (local address 127.0.253.195:39525)
0812 16:37:09.000474 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:09.000476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:09.000596 (+ 120us) server_negotiation.cc:407] Connection header received
0812 16:37:09.000628 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:09.000629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:09.000646 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:09.000704 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:37:09.000949 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:09.001697 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:09.002596 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:09.002905 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:09.002974 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:09.003225 (+ 251us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:37:09.003325 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:37:09.003400 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:09.003460 (+ 60us) server_negotiation.cc:299] Negotiation successful
0812 16:37:09.003481 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 16:37:09.008625 5433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:09.006276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:32815 (local address 127.0.253.194:37559)
0812 16:37:09.006340 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:09.006342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:09.006352 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:37:09.006387 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:09.006388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:09.006398 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:09.006444 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:37:09.006610 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:09.007142 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:09.007874 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:09.008085 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:09.008164 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:09.008345 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:37:09.008378 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:37:09.008453 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:09.008536 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 16:37:09.008556 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 16:37:09.008633 5403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:09.006176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:37559 (local address 127.0.253.195:32815)
0812 16:37:09.006266 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:09.006276 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:09.006332 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:09.006446 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:09.006446 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:09.006447 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 16:37:09.006509 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:09.006510 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:09.007222 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:09.007223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:09.007790 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:09.007792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:09.008166 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:09.008168 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:09.008235 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:09.008249 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 16:37:09.008474 (+ 225us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 16:37:09.008476 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 16:37:09.008522 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 16:37:09.008568 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260812 16:37:09.011382 4824 raft_consensus.cc:1261] T fc79fb036cec47f5b949b37573764fbb P b36bcb39624f44849c3b9ccfadfbf34d [term 1 FOLLOWER]: Refusing update from remote peer e1f11bd30d53435680b94197726c74bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:37:09.011394 5075 raft_consensus.cc:1261] T fc79fb036cec47f5b949b37573764fbb P 7133565a63d24fe09d46b0f58def254b [term 1 FOLLOWER]: Refusing update from remote peer e1f11bd30d53435680b94197726c74bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:37:09.011696 5476 consensus_queue.cc:1035] T fc79fb036cec47f5b949b37573764fbb P e1f11bd30d53435680b94197726c74bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:37:09.011864 5472 consensus_queue.cc:1035] T fc79fb036cec47f5b949b37573764fbb P e1f11bd30d53435680b94197726c74bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:37:09.013545 5503 mvcc.cc:205] Tried to move back new op lower bound from 7317719568428281856 to 7317719567797088256. Current Snapshot: MvccSnapshot[applied={T|T < 7317719568428281856}]
I20260812 16:37:09.021337 5436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:09.018498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:41397 (local address 127.0.253.195:39525)
0812 16:37:09.018537 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:09.018538 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:09.018545 (+ 7us) server_negotiation.cc:407] Connection header received
0812 16:37:09.018569 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:09.018569 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:09.018580 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:09.018627 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:37:09.018800 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:09.019576 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:09.020596 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:09.020826 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:09.020904 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:09.021092 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:37:09.021127 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:37:09.021178 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:09.021241 (+ 63us) server_negotiation.cc:299] Negotiation successful
0812 16:37:09.021262 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 16:37:09.021343 5183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:09.018274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.195:39525 (local address 127.0.253.194:41397)
0812 16:37:09.018371 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:09.018383 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:09.018428 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:09.018647 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:09.018648 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:09.018650 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 16:37:09.018726 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:09.018727 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:09.019670 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:09.019671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:09.020517 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:09.020519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:09.020902 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:09.020903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:09.020969 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:09.020994 (+ 25us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 16:37:09.021185 (+ 191us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 16:37:09.021186 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 16:37:09.021237 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 16:37:09.021279 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260812 16:37:09.031284 4525 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38380:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:37:09.031409 4525 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38380:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:37:09.032210 4525 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 028fd69b0c7645bca0e07c1476188e82: Sending DeleteTablet for 3 replicas of tablet fc79fb036cec47f5b949b37573764fbb
I20260812 16:37:09.032560 5296 tablet_service.cc:1514] Processing DeleteTablet for tablet fc79fb036cec47f5b949b37573764fbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:09 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:41812
I20260812 16:37:09.032559 4790 tablet_service.cc:1514] Processing DeleteTablet for tablet fc79fb036cec47f5b949b37573764fbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:09 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:51386
I20260812 16:37:09.032936 5039 tablet_service.cc:1514] Processing DeleteTablet for tablet fc79fb036cec47f5b949b37573764fbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:09 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:59048
I20260812 16:37:09.033181 4525 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38380:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:37:09.033342 4525 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:37:09.033649 5532 tablet_replica.cc:330] T fc79fb036cec47f5b949b37573764fbb P e1f11bd30d53435680b94197726c74bf: stopping tablet replica
I20260812 16:37:09.033763 5532 raft_consensus.cc:2229] T fc79fb036cec47f5b949b37573764fbb P e1f11bd30d53435680b94197726c74bf [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:37:09.033864 5532 raft_consensus.cc:2258] T fc79fb036cec47f5b949b37573764fbb P e1f11bd30d53435680b94197726c74bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:09.034735 5296 tablet_service.cc:1467] Processing CreateTablet for tablet 9cedf60b4b014730b505b16705cf8d01 (DEFAULT_TABLE table=test-table [id=c796aea705ff4c6880f7d18914c29266]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:09.034735 5039 tablet_service.cc:1467] Processing CreateTablet for tablet 9cedf60b4b014730b505b16705cf8d01 (DEFAULT_TABLE table=test-table [id=c796aea705ff4c6880f7d18914c29266]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:09.034750 4790 tablet_service.cc:1467] Processing CreateTablet for tablet 9cedf60b4b014730b505b16705cf8d01 (DEFAULT_TABLE table=test-table [id=c796aea705ff4c6880f7d18914c29266]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:09.034874 5296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cedf60b4b014730b505b16705cf8d01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:09.034873 5039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cedf60b4b014730b505b16705cf8d01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:09.034884 4790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cedf60b4b014730b505b16705cf8d01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:09.036321 5445 tablet_bootstrap.cc:492] T 9cedf60b4b014730b505b16705cf8d01 P b36bcb39624f44849c3b9ccfadfbf34d: Bootstrap starting.
I20260812 16:37:09.036329 5439 tablet_bootstrap.cc:492] T 9cedf60b4b014730b505b16705cf8d01 P 7133565a63d24fe09d46b0f58def254b: Bootstrap starting.
I20260812 16:37:09.036345 5448 tablet_bootstrap.cc:492] T 9cedf60b4b014730b505b16705cf8d01 P e1f11bd30d53435680b94197726c74bf: Bootstrap starting.
I20260812 16:37:09.036742 5445 tablet_bootstrap.cc:654] T 9cedf60b4b014730b505b16705cf8d01 P b36bcb39624f44849c3b9ccfadfbf34d: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:09.036748 5448 tablet_bootstrap.cc:654] T 9cedf60b4b014730b505b16705cf8d01 P e1f11bd30d53435680b94197726c74bf: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:09.036747 5439 tablet_bootstrap.cc:654] T 9cedf60b4b014730b505b16705cf8d01 P 7133565a63d24fe09d46b0f58def254b: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:09.037454 5532 ts_tablet_manager.cc:1905] T fc79fb036cec47f5b949b37573764fbb P e1f11bd30d53435680b94197726c74bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:37:09.037583 5439 tablet_bootstrap.cc:492] T 9cedf60b4b014730b505b16705cf8d01 P 7133565a63d24fe09d46b0f58def254b: No bootstrap required, opened a new log
I20260812 16:37:09.037638 5439 ts_tablet_manager.cc:1397] T 9cedf60b4b014730b505b16705cf8d01 P 7133565a63d24fe09d46b0f58def254b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:09.037822 5439 raft_consensus.cc:348] T 9cedf60b4b014730b505b16705cf8d01 P 7133565a63d24fe09d46b0f58def254b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:09.037906 5439 raft_consensus.cc:374] T 9cedf60b4b014730b505b16705cf8d01 P 7133565a63d24fe09d46b0f58def254b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:09.037930 5439 raft_consensus.cc:729] T 9cedf60b4b014730b505b16705cf8d01 P 7133565a63d24fe09d46b0f58def254b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7133565a63d24fe09d46b0f58def254b, State: Initialized, Role: FOLLOWER
I20260812 16:37:09.037976 5439 consensus_queue.cc:260] T 9cedf60b4b014730b505b16705cf8d01 P 7133565a63d24fe09d46b0f58def254b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:09.038075 5439 ts_tablet_manager.cc:1428] T 9cedf60b4b014730b505b16705cf8d01 P 7133565a63d24fe09d46b0f58def254b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 16:37:09.039326 5532 ts_tablet_manager.cc:1918] T fc79fb036cec47f5b949b37573764fbb P e1f11bd30d53435680b94197726c74bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:37:09.039400 5532 log.cc:1196] T fc79fb036cec47f5b949b37573764fbb P e1f11bd30d53435680b94197726c74bf: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-2/wal/wals/fc79fb036cec47f5b949b37573764fbb
I20260812 16:37:09.039671 5532 ts_tablet_manager.cc:1939] T fc79fb036cec47f5b949b37573764fbb P e1f11bd30d53435680b94197726c74bf: Deleting consensus metadata
I20260812 16:37:09.040105 4496 catalog_manager.cc:4841] TS e1f11bd30d53435680b94197726c74bf (127.0.253.195:39525): tablet fc79fb036cec47f5b949b37573764fbb (table test-table [id=488af6b45b55462982695c5698cbc178]) successfully deleted
I20260812 16:37:09.040143 5448 tablet_bootstrap.cc:492] T 9cedf60b4b014730b505b16705cf8d01 P e1f11bd30d53435680b94197726c74bf: No bootstrap required, opened a new log
I20260812 16:37:09.040192 5448 ts_tablet_manager.cc:1397] T 9cedf60b4b014730b505b16705cf8d01 P e1f11bd30d53435680b94197726c74bf: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260812 16:37:09.040337 5448 raft_consensus.cc:348] T 9cedf60b4b014730b505b16705cf8d01 P e1f11bd30d53435680b94197726c74bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:09.040406 5448 raft_consensus.cc:374] T 9cedf60b4b014730b505b16705cf8d01 P e1f11bd30d53435680b94197726c74bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:09.040433 5448 raft_consensus.cc:729] T 9cedf60b4b014730b505b16705cf8d01 P e1f11bd30d53435680b94197726c74bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f11bd30d53435680b94197726c74bf, State: Initialized, Role: FOLLOWER
I20260812 16:37:09.040478 5448 consensus_queue.cc:260] T 9cedf60b4b014730b505b16705cf8d01 P e1f11bd30d53435680b94197726c74bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:09.040558 5448 ts_tablet_manager.cc:1428] T 9cedf60b4b014730b505b16705cf8d01 P e1f11bd30d53435680b94197726c74bf: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 16:37:09.043215 5535 tablet_replica.cc:330] T fc79fb036cec47f5b949b37573764fbb P 7133565a63d24fe09d46b0f58def254b: stopping tablet replica
I20260812 16:37:09.043376 5534 tablet_replica.cc:330] T fc79fb036cec47f5b949b37573764fbb P b36bcb39624f44849c3b9ccfadfbf34d: stopping tablet replica
I20260812 16:37:09.043512 5445 tablet_bootstrap.cc:492] T 9cedf60b4b014730b505b16705cf8d01 P b36bcb39624f44849c3b9ccfadfbf34d: No bootstrap required, opened a new log
I20260812 16:37:09.043669 5535 raft_consensus.cc:2229] T fc79fb036cec47f5b949b37573764fbb P 7133565a63d24fe09d46b0f58def254b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:37:09.043751 5535 raft_consensus.cc:2258] T fc79fb036cec47f5b949b37573764fbb P 7133565a63d24fe09d46b0f58def254b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:09.043576 5445 ts_tablet_manager.cc:1397] T 9cedf60b4b014730b505b16705cf8d01 P b36bcb39624f44849c3b9ccfadfbf34d: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260812 16:37:09.044020 5445 raft_consensus.cc:348] T 9cedf60b4b014730b505b16705cf8d01 P b36bcb39624f44849c3b9ccfadfbf34d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:09.044099 5445 raft_consensus.cc:374] T 9cedf60b4b014730b505b16705cf8d01 P b36bcb39624f44849c3b9ccfadfbf34d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:09.044126 5445 raft_consensus.cc:729] T 9cedf60b4b014730b505b16705cf8d01 P b36bcb39624f44849c3b9ccfadfbf34d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b36bcb39624f44849c3b9ccfadfbf34d, State: Initialized, Role: FOLLOWER
I20260812 16:37:09.044195 5445 consensus_queue.cc:260] T 9cedf60b4b014730b505b16705cf8d01 P b36bcb39624f44849c3b9ccfadfbf34d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:09.044284 5445 ts_tablet_manager.cc:1428] T 9cedf60b4b014730b505b16705cf8d01 P b36bcb39624f44849c3b9ccfadfbf34d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:09.044456 5535 ts_tablet_manager.cc:1905] T fc79fb036cec47f5b949b37573764fbb P 7133565a63d24fe09d46b0f58def254b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:37:09.043550 5534 raft_consensus.cc:2229] T fc79fb036cec47f5b949b37573764fbb P b36bcb39624f44849c3b9ccfadfbf34d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:37:09.045090 5534 raft_consensus.cc:2258] T fc79fb036cec47f5b949b37573764fbb P b36bcb39624f44849c3b9ccfadfbf34d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:09.045562 5534 ts_tablet_manager.cc:1905] T fc79fb036cec47f5b949b37573764fbb P b36bcb39624f44849c3b9ccfadfbf34d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:37:09.046279 5535 ts_tablet_manager.cc:1918] T fc79fb036cec47f5b949b37573764fbb P 7133565a63d24fe09d46b0f58def254b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:37:09.046409 5535 log.cc:1196] T fc79fb036cec47f5b949b37573764fbb P 7133565a63d24fe09d46b0f58def254b: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-1/wal/wals/fc79fb036cec47f5b949b37573764fbb
I20260812 16:37:09.046732 5535 ts_tablet_manager.cc:1939] T fc79fb036cec47f5b949b37573764fbb P 7133565a63d24fe09d46b0f58def254b: Deleting consensus metadata
I20260812 16:37:09.047170 5534 ts_tablet_manager.cc:1918] T fc79fb036cec47f5b949b37573764fbb P b36bcb39624f44849c3b9ccfadfbf34d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:37:09.047247 5534 log.cc:1196] T fc79fb036cec47f5b949b37573764fbb P b36bcb39624f44849c3b9ccfadfbf34d: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-0/wal/wals/fc79fb036cec47f5b949b37573764fbb
I20260812 16:37:09.047294 4497 catalog_manager.cc:4841] TS 7133565a63d24fe09d46b0f58def254b (127.0.253.194:37559): tablet fc79fb036cec47f5b949b37573764fbb (table test-table [id=488af6b45b55462982695c5698cbc178]) successfully deleted
I20260812 16:37:09.047516 5534 ts_tablet_manager.cc:1939] T fc79fb036cec47f5b949b37573764fbb P b36bcb39624f44849c3b9ccfadfbf34d: Deleting consensus metadata
I20260812 16:37:09.047964 4496 catalog_manager.cc:4841] TS b36bcb39624f44849c3b9ccfadfbf34d (127.0.253.193:46443): tablet fc79fb036cec47f5b949b37573764fbb (table test-table [id=488af6b45b55462982695c5698cbc178]) successfully deleted
I20260812 16:37:09.295014 5527 raft_consensus.cc:482] T 9cedf60b4b014730b505b16705cf8d01 P 7133565a63d24fe09d46b0f58def254b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:37:09.295198 5527 raft_consensus.cc:504] T 9cedf60b4b014730b505b16705cf8d01 P 7133565a63d24fe09d46b0f58def254b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:09.295359 5527 leader_election.cc:288] T 9cedf60b4b014730b505b16705cf8d01 P 7133565a63d24fe09d46b0f58def254b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b36bcb39624f44849c3b9ccfadfbf34d (127.0.253.193:46443), e1f11bd30d53435680b94197726c74bf (127.0.253.195:39525)
I20260812 16:37:09.295697 4824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cedf60b4b014730b505b16705cf8d01" candidate_uuid: "7133565a63d24fe09d46b0f58def254b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" is_pre_election: true
I20260812 16:37:09.295698 5324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cedf60b4b014730b505b16705cf8d01" candidate_uuid: "7133565a63d24fe09d46b0f58def254b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f11bd30d53435680b94197726c74bf" is_pre_election: true
I20260812 16:37:09.295796 4824 raft_consensus.cc:2454] T 9cedf60b4b014730b505b16705cf8d01 P b36bcb39624f44849c3b9ccfadfbf34d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7133565a63d24fe09d46b0f58def254b in term 0.
I20260812 16:37:09.295826 5324 raft_consensus.cc:2454] T 9cedf60b4b014730b505b16705cf8d01 P e1f11bd30d53435680b94197726c74bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7133565a63d24fe09d46b0f58def254b in term 0.
I20260812 16:37:09.295979 4961 leader_election.cc:302] T 9cedf60b4b014730b505b16705cf8d01 P 7133565a63d24fe09d46b0f58def254b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7133565a63d24fe09d46b0f58def254b, b36bcb39624f44849c3b9ccfadfbf34d; no voters:
I20260812 16:37:09.296101 5527 raft_consensus.cc:2781] T 9cedf60b4b014730b505b16705cf8d01 P 7133565a63d24fe09d46b0f58def254b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:37:09.296157 5527 raft_consensus.cc:482] T 9cedf60b4b014730b505b16705cf8d01 P 7133565a63d24fe09d46b0f58def254b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:37:09.296172 5527 raft_consensus.cc:3037] T 9cedf60b4b014730b505b16705cf8d01 P 7133565a63d24fe09d46b0f58def254b [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:09.296986 5527 raft_consensus.cc:504] T 9cedf60b4b014730b505b16705cf8d01 P 7133565a63d24fe09d46b0f58def254b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:09.297096 5527 leader_election.cc:288] T 9cedf60b4b014730b505b16705cf8d01 P 7133565a63d24fe09d46b0f58def254b [CANDIDATE]: Term 1 election: Requested vote from peers b36bcb39624f44849c3b9ccfadfbf34d (127.0.253.193:46443), e1f11bd30d53435680b94197726c74bf (127.0.253.195:39525)
I20260812 16:37:09.297322 4824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cedf60b4b014730b505b16705cf8d01" candidate_uuid: "7133565a63d24fe09d46b0f58def254b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b36bcb39624f44849c3b9ccfadfbf34d"
I20260812 16:37:09.297358 5324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cedf60b4b014730b505b16705cf8d01" candidate_uuid: "7133565a63d24fe09d46b0f58def254b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f11bd30d53435680b94197726c74bf"
I20260812 16:37:09.297436 5324 raft_consensus.cc:3037] T 9cedf60b4b014730b505b16705cf8d01 P e1f11bd30d53435680b94197726c74bf [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:09.297433 4824 raft_consensus.cc:3037] T 9cedf60b4b014730b505b16705cf8d01 P b36bcb39624f44849c3b9ccfadfbf34d [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:09.298096 5324 raft_consensus.cc:2454] T 9cedf60b4b014730b505b16705cf8d01 P e1f11bd30d53435680b94197726c74bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7133565a63d24fe09d46b0f58def254b in term 1.
I20260812 16:37:09.298241 4824 raft_consensus.cc:2454] T 9cedf60b4b014730b505b16705cf8d01 P b36bcb39624f44849c3b9ccfadfbf34d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7133565a63d24fe09d46b0f58def254b in term 1.
I20260812 16:37:09.298295 4961 leader_election.cc:302] T 9cedf60b4b014730b505b16705cf8d01 P 7133565a63d24fe09d46b0f58def254b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7133565a63d24fe09d46b0f58def254b, e1f11bd30d53435680b94197726c74bf; no voters:
I20260812 16:37:09.298375 5527 raft_consensus.cc:2781] T 9cedf60b4b014730b505b16705cf8d01 P 7133565a63d24fe09d46b0f58def254b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:37:09.298431 5527 raft_consensus.cc:686] T 9cedf60b4b014730b505b16705cf8d01 P 7133565a63d24fe09d46b0f58def254b [term 1 LEADER]: Becoming Leader. State: Replica: 7133565a63d24fe09d46b0f58def254b, State: Running, Role: LEADER
I20260812 16:37:09.298486 5527 consensus_queue.cc:237] T 9cedf60b4b014730b505b16705cf8d01 P 7133565a63d24fe09d46b0f58def254b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:09.299250 4525 catalog_manager.cc:5495] T 9cedf60b4b014730b505b16705cf8d01 P 7133565a63d24fe09d46b0f58def254b reported cstate change: term changed from 0 to 1, leader changed from <none> to 7133565a63d24fe09d46b0f58def254b (127.0.253.194). New cstate: current_term: 1 leader_uuid: "7133565a63d24fe09d46b0f58def254b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } health_report { overall_health: UNKNOWN } } }
I20260812 16:37:09.325053 5433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:09.322984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59076 (local address 127.0.253.194:37559)
0812 16:37:09.323026 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:09.323028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:09.323062 (+ 34us) server_negotiation.cc:407] Connection header received
0812 16:37:09.323143 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:09.323144 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:09.323153 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:09.323203 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:37:09.323316 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:09.323777 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:09.324488 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:09.324712 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:09.324764 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:09.324876 (+ 112us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:37:09.324901 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:37:09.324934 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:09.324982 (+ 48us) server_negotiation.cc:299] Negotiation successful
0812 16:37:09.324996 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 16:37:09.329136 5183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:09.327052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:37559 (local address 127.0.253.194:60447)
0812 16:37:09.327108 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:09.327118 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:09.327162 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:09.327265 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:09.327266 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:09.327267 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 16:37:09.327313 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:09.327314 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:09.327883 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:09.327884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:09.328483 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:09.328484 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:09.328825 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:09.328826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:09.328865 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:09.328868 (+ 3us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 16:37:09.329005 (+ 137us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 16:37:09.329006 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 16:37:09.329038 (+ 32us) client_negotiation.cc:240] Negotiation successful
0812 16:37:09.329069 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 16:37:09.329144 5433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:09.327126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:60447 (local address 127.0.253.194:37559)
0812 16:37:09.327164 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:09.327166 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:09.327180 (+ 14us) server_negotiation.cc:407] Connection header received
0812 16:37:09.327217 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:09.327218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:09.327225 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:09.327264 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:37:09.327366 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:09.327805 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:09.328558 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:09.328743 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:09.328797 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:09.328931 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:37:09.328951 (+ 20us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:37:09.328989 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:09.329045 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 16:37:09.329058 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 16:37:09.331212 5324 raft_consensus.cc:1261] T 9cedf60b4b014730b505b16705cf8d01 P e1f11bd30d53435680b94197726c74bf [term 1 FOLLOWER]: Refusing update from remote peer 7133565a63d24fe09d46b0f58def254b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:37:09.331260 4824 raft_consensus.cc:1261] T 9cedf60b4b014730b505b16705cf8d01 P b36bcb39624f44849c3b9ccfadfbf34d [term 1 FOLLOWER]: Refusing update from remote peer 7133565a63d24fe09d46b0f58def254b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:37:09.331466 5494 consensus_queue.cc:1035] T 9cedf60b4b014730b505b16705cf8d01 P 7133565a63d24fe09d46b0f58def254b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:37:09.331559 5527 consensus_queue.cc:1035] T 9cedf60b4b014730b505b16705cf8d01 P 7133565a63d24fe09d46b0f58def254b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 16:37:09.336205 5503 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 16:37:10.856211 4526 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38380:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:37:10.856314 4526 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38380:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:37:10.857206 4526 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 028fd69b0c7645bca0e07c1476188e82: Sending DeleteTablet for 3 replicas of tablet 9cedf60b4b014730b505b16705cf8d01
I20260812 16:37:10.857527 5296 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cedf60b4b014730b505b16705cf8d01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:10 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:41812
I20260812 16:37:10.857524 4790 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cedf60b4b014730b505b16705cf8d01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:10 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:51386
I20260812 16:37:10.857744 5039 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cedf60b4b014730b505b16705cf8d01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:10 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:59048
I20260812 16:37:10.857937 5591 tablet_replica.cc:330] T 9cedf60b4b014730b505b16705cf8d01 P 7133565a63d24fe09d46b0f58def254b: stopping tablet replica
I20260812 16:37:10.857955 5583 tablet_replica.cc:330] T 9cedf60b4b014730b505b16705cf8d01 P e1f11bd30d53435680b94197726c74bf: stopping tablet replica
I20260812 16:37:10.858006 5583 raft_consensus.cc:2229] T 9cedf60b4b014730b505b16705cf8d01 P e1f11bd30d53435680b94197726c74bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:37:10.858011 5591 raft_consensus.cc:2229] T 9cedf60b4b014730b505b16705cf8d01 P 7133565a63d24fe09d46b0f58def254b [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:37:10.858045 5583 raft_consensus.cc:2258] T 9cedf60b4b014730b505b16705cf8d01 P e1f11bd30d53435680b94197726c74bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:10.858095 5591 raft_consensus.cc:2258] T 9cedf60b4b014730b505b16705cf8d01 P 7133565a63d24fe09d46b0f58def254b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:10.858289 5583 ts_tablet_manager.cc:1905] T 9cedf60b4b014730b505b16705cf8d01 P e1f11bd30d53435680b94197726c74bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:37:10.859927 5583 ts_tablet_manager.cc:1918] T 9cedf60b4b014730b505b16705cf8d01 P e1f11bd30d53435680b94197726c74bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 16:37:10.860002 5583 log.cc:1196] T 9cedf60b4b014730b505b16705cf8d01 P e1f11bd30d53435680b94197726c74bf: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-2/wal/wals/9cedf60b4b014730b505b16705cf8d01
I20260812 16:37:10.860241 5583 ts_tablet_manager.cc:1939] T 9cedf60b4b014730b505b16705cf8d01 P e1f11bd30d53435680b94197726c74bf: Deleting consensus metadata
I20260812 16:37:10.860579 5587 tablet_replica.cc:330] T 9cedf60b4b014730b505b16705cf8d01 P b36bcb39624f44849c3b9ccfadfbf34d: stopping tablet replica
I20260812 16:37:10.860606 4496 catalog_manager.cc:4841] TS e1f11bd30d53435680b94197726c74bf (127.0.253.195:39525): tablet 9cedf60b4b014730b505b16705cf8d01 (table test-table [id=c796aea705ff4c6880f7d18914c29266]) successfully deleted
I20260812 16:37:10.860648 5587 raft_consensus.cc:2229] T 9cedf60b4b014730b505b16705cf8d01 P b36bcb39624f44849c3b9ccfadfbf34d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:37:10.860688 5587 raft_consensus.cc:2258] T 9cedf60b4b014730b505b16705cf8d01 P b36bcb39624f44849c3b9ccfadfbf34d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:10.860733 5591 ts_tablet_manager.cc:1905] T 9cedf60b4b014730b505b16705cf8d01 P 7133565a63d24fe09d46b0f58def254b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:37:10.860903 5587 ts_tablet_manager.cc:1905] T 9cedf60b4b014730b505b16705cf8d01 P b36bcb39624f44849c3b9ccfadfbf34d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:37:10.862145 5587 ts_tablet_manager.cc:1918] T 9cedf60b4b014730b505b16705cf8d01 P b36bcb39624f44849c3b9ccfadfbf34d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 16:37:10.862177 5591 ts_tablet_manager.cc:1918] T 9cedf60b4b014730b505b16705cf8d01 P 7133565a63d24fe09d46b0f58def254b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 16:37:10.862222 5591 log.cc:1196] T 9cedf60b4b014730b505b16705cf8d01 P 7133565a63d24fe09d46b0f58def254b: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-1/wal/wals/9cedf60b4b014730b505b16705cf8d01
I20260812 16:37:10.862228 5587 log.cc:1196] T 9cedf60b4b014730b505b16705cf8d01 P b36bcb39624f44849c3b9ccfadfbf34d: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-0/wal/wals/9cedf60b4b014730b505b16705cf8d01
I20260812 16:37:10.862458 5587 ts_tablet_manager.cc:1939] T 9cedf60b4b014730b505b16705cf8d01 P b36bcb39624f44849c3b9ccfadfbf34d: Deleting consensus metadata
I20260812 16:37:10.862572 5591 ts_tablet_manager.cc:1939] T 9cedf60b4b014730b505b16705cf8d01 P 7133565a63d24fe09d46b0f58def254b: Deleting consensus metadata
I20260812 16:37:10.862787 4496 catalog_manager.cc:4841] TS b36bcb39624f44849c3b9ccfadfbf34d (127.0.253.193:46443): tablet 9cedf60b4b014730b505b16705cf8d01 (table test-table [id=c796aea705ff4c6880f7d18914c29266]) successfully deleted
I20260812 16:37:10.862912 4497 catalog_manager.cc:4841] TS 7133565a63d24fe09d46b0f58def254b (127.0.253.194:37559): tablet 9cedf60b4b014730b505b16705cf8d01 (table test-table [id=c796aea705ff4c6880f7d18914c29266]) successfully deleted
Aug 12 16:37:10 dist-test-slave-w4v5 krb5kdc[4447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552630, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 16:37:10 dist-test-slave-w4v5 krb5kdc[4447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552630, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:10.885202 5605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:10.877427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38390 (local address 127.0.253.254:41125)
0812 16:37:10.877587 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:10.877590 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:10.877682 (+ 92us) server_negotiation.cc:407] Connection header received
0812 16:37:10.877788 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:10.877789 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:10.877802 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:10.877893 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:10.878464 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:10.878992 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:10.879779 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:10.879996 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:10.880092 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:10.882436 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:10.882437 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:10.882438 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:10.882446 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:10.883971 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:10.884348 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:10.884350 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:10.884350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:10.884412 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:10.884641 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:10.884642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:10.884642 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:10.884770 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:10.884879 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:10.884985 (+ 106us) server_negotiation.cc:299] Negotiation successful
0812 16:37:10.885078 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":79,"threads_started":1}
I20260812 16:37:10.886858 4526 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38390:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:37:10.887071 4526 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:37:10.888657 4790 tablet_service.cc:1467] Processing CreateTablet for tablet 13c3aa3d51b145dd8f19c87f3dcc4a8f (DEFAULT_TABLE table=test-table [id=4d03b298d4ea496c9a569f04b09aaf19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:10.888657 5039 tablet_service.cc:1467] Processing CreateTablet for tablet 13c3aa3d51b145dd8f19c87f3dcc4a8f (DEFAULT_TABLE table=test-table [id=4d03b298d4ea496c9a569f04b09aaf19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:10.888809 4790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13c3aa3d51b145dd8f19c87f3dcc4a8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:10.888811 5039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13c3aa3d51b145dd8f19c87f3dcc4a8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:10.889847 5606 tablet_bootstrap.cc:492] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P b36bcb39624f44849c3b9ccfadfbf34d: Bootstrap starting.
I20260812 16:37:10.890003 5607 tablet_bootstrap.cc:492] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P 7133565a63d24fe09d46b0f58def254b: Bootstrap starting.
I20260812 16:37:10.890014 5296 tablet_service.cc:1467] Processing CreateTablet for tablet 13c3aa3d51b145dd8f19c87f3dcc4a8f (DEFAULT_TABLE table=test-table [id=4d03b298d4ea496c9a569f04b09aaf19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:10.890164 5296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13c3aa3d51b145dd8f19c87f3dcc4a8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:10.890305 5606 tablet_bootstrap.cc:654] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P b36bcb39624f44849c3b9ccfadfbf34d: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:10.890442 5607 tablet_bootstrap.cc:654] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P 7133565a63d24fe09d46b0f58def254b: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:10.891108 5607 tablet_bootstrap.cc:492] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P 7133565a63d24fe09d46b0f58def254b: No bootstrap required, opened a new log
I20260812 16:37:10.891173 5607 ts_tablet_manager.cc:1397] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P 7133565a63d24fe09d46b0f58def254b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:10.891304 5607 raft_consensus.cc:348] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P 7133565a63d24fe09d46b0f58def254b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:10.891386 5607 raft_consensus.cc:374] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P 7133565a63d24fe09d46b0f58def254b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:10.891408 5607 raft_consensus.cc:729] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P 7133565a63d24fe09d46b0f58def254b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7133565a63d24fe09d46b0f58def254b, State: Initialized, Role: FOLLOWER
I20260812 16:37:10.891461 5607 consensus_queue.cc:260] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P 7133565a63d24fe09d46b0f58def254b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:10.891568 5607 ts_tablet_manager.cc:1428] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P 7133565a63d24fe09d46b0f58def254b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:10.891505 5606 tablet_bootstrap.cc:492] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P b36bcb39624f44849c3b9ccfadfbf34d: No bootstrap required, opened a new log
I20260812 16:37:10.891618 5606 ts_tablet_manager.cc:1397] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P b36bcb39624f44849c3b9ccfadfbf34d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:37:10.891789 5616 tablet_bootstrap.cc:492] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P e1f11bd30d53435680b94197726c74bf: Bootstrap starting.
I20260812 16:37:10.891814 5606 raft_consensus.cc:348] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P b36bcb39624f44849c3b9ccfadfbf34d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:10.891902 5606 raft_consensus.cc:374] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P b36bcb39624f44849c3b9ccfadfbf34d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:10.891933 5606 raft_consensus.cc:729] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P b36bcb39624f44849c3b9ccfadfbf34d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b36bcb39624f44849c3b9ccfadfbf34d, State: Initialized, Role: FOLLOWER
I20260812 16:37:10.891999 5606 consensus_queue.cc:260] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P b36bcb39624f44849c3b9ccfadfbf34d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:10.892217 5606 ts_tablet_manager.cc:1428] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P b36bcb39624f44849c3b9ccfadfbf34d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:10.892575 5616 tablet_bootstrap.cc:654] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P e1f11bd30d53435680b94197726c74bf: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:10.893244 5616 tablet_bootstrap.cc:492] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P e1f11bd30d53435680b94197726c74bf: No bootstrap required, opened a new log
I20260812 16:37:10.893312 5616 ts_tablet_manager.cc:1397] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P e1f11bd30d53435680b94197726c74bf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:37:10.893435 5616 raft_consensus.cc:348] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P e1f11bd30d53435680b94197726c74bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:10.893510 5616 raft_consensus.cc:374] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P e1f11bd30d53435680b94197726c74bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:10.893543 5616 raft_consensus.cc:729] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P e1f11bd30d53435680b94197726c74bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f11bd30d53435680b94197726c74bf, State: Initialized, Role: FOLLOWER
I20260812 16:37:10.893596 5616 consensus_queue.cc:260] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P e1f11bd30d53435680b94197726c74bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:10.893777 5616 ts_tablet_manager.cc:1428] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P e1f11bd30d53435680b94197726c74bf: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 16:37:10.965794 5564 raft_consensus.cc:482] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P 7133565a63d24fe09d46b0f58def254b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:37:10.965896 5564 raft_consensus.cc:504] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P 7133565a63d24fe09d46b0f58def254b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:10.966056 5564 leader_election.cc:288] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P 7133565a63d24fe09d46b0f58def254b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b36bcb39624f44849c3b9ccfadfbf34d (127.0.253.193:46443), e1f11bd30d53435680b94197726c74bf (127.0.253.195:39525)
I20260812 16:37:10.966290 4824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c3aa3d51b145dd8f19c87f3dcc4a8f" candidate_uuid: "7133565a63d24fe09d46b0f58def254b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" is_pre_election: true
I20260812 16:37:10.966379 4824 raft_consensus.cc:2454] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P b36bcb39624f44849c3b9ccfadfbf34d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7133565a63d24fe09d46b0f58def254b in term 0.
I20260812 16:37:10.966331 5324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c3aa3d51b145dd8f19c87f3dcc4a8f" candidate_uuid: "7133565a63d24fe09d46b0f58def254b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f11bd30d53435680b94197726c74bf" is_pre_election: true
I20260812 16:37:10.966476 5324 raft_consensus.cc:2454] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P e1f11bd30d53435680b94197726c74bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7133565a63d24fe09d46b0f58def254b in term 0.
I20260812 16:37:10.966553 4961 leader_election.cc:302] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P 7133565a63d24fe09d46b0f58def254b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7133565a63d24fe09d46b0f58def254b, b36bcb39624f44849c3b9ccfadfbf34d; no voters:
I20260812 16:37:10.966665 5564 raft_consensus.cc:2781] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P 7133565a63d24fe09d46b0f58def254b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:37:10.966742 5564 raft_consensus.cc:482] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P 7133565a63d24fe09d46b0f58def254b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:37:10.966779 5564 raft_consensus.cc:3037] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P 7133565a63d24fe09d46b0f58def254b [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:10.967751 5564 raft_consensus.cc:504] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P 7133565a63d24fe09d46b0f58def254b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:10.967926 5564 leader_election.cc:288] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P 7133565a63d24fe09d46b0f58def254b [CANDIDATE]: Term 1 election: Requested vote from peers b36bcb39624f44849c3b9ccfadfbf34d (127.0.253.193:46443), e1f11bd30d53435680b94197726c74bf (127.0.253.195:39525)
I20260812 16:37:10.968189 5324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c3aa3d51b145dd8f19c87f3dcc4a8f" candidate_uuid: "7133565a63d24fe09d46b0f58def254b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f11bd30d53435680b94197726c74bf"
I20260812 16:37:10.968163 4824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13c3aa3d51b145dd8f19c87f3dcc4a8f" candidate_uuid: "7133565a63d24fe09d46b0f58def254b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b36bcb39624f44849c3b9ccfadfbf34d"
I20260812 16:37:10.968276 5324 raft_consensus.cc:3037] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P e1f11bd30d53435680b94197726c74bf [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:10.968313 4824 raft_consensus.cc:3037] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P b36bcb39624f44849c3b9ccfadfbf34d [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:10.968958 5324 raft_consensus.cc:2454] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P e1f11bd30d53435680b94197726c74bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7133565a63d24fe09d46b0f58def254b in term 1.
I20260812 16:37:10.969187 4824 raft_consensus.cc:2454] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P b36bcb39624f44849c3b9ccfadfbf34d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7133565a63d24fe09d46b0f58def254b in term 1.
I20260812 16:37:10.969193 4961 leader_election.cc:302] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P 7133565a63d24fe09d46b0f58def254b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7133565a63d24fe09d46b0f58def254b, e1f11bd30d53435680b94197726c74bf; no voters:
I20260812 16:37:10.969308 5564 raft_consensus.cc:2781] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P 7133565a63d24fe09d46b0f58def254b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:37:10.969360 5564 raft_consensus.cc:686] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P 7133565a63d24fe09d46b0f58def254b [term 1 LEADER]: Becoming Leader. State: Replica: 7133565a63d24fe09d46b0f58def254b, State: Running, Role: LEADER
I20260812 16:37:10.969422 5564 consensus_queue.cc:237] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P 7133565a63d24fe09d46b0f58def254b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:10.970177 4526 catalog_manager.cc:5495] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P 7133565a63d24fe09d46b0f58def254b reported cstate change: term changed from 0 to 1, leader changed from <none> to 7133565a63d24fe09d46b0f58def254b (127.0.253.194). New cstate: current_term: 1 leader_uuid: "7133565a63d24fe09d46b0f58def254b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } health_report { overall_health: UNKNOWN } } }
I20260812 16:37:10.988725 5627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:10.985961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59080 (local address 127.0.253.194:37559)
0812 16:37:10.986167 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:10.986168 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:10.986175 (+ 7us) server_negotiation.cc:407] Connection header received
0812 16:37:10.986195 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:10.986196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:10.986206 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:10.986291 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:37:10.986466 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:10.987013 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:10.987906 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:10.988146 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:10.988232 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:10.988410 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:37:10.988454 (+ 44us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:37:10.988524 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:10.988594 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 16:37:10.988617 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":106,"threads_started":1}
I20260812 16:37:10.991358 5324 raft_consensus.cc:1261] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P e1f11bd30d53435680b94197726c74bf [term 1 FOLLOWER]: Refusing update from remote peer 7133565a63d24fe09d46b0f58def254b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:37:10.991379 4824 raft_consensus.cc:1261] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P b36bcb39624f44849c3b9ccfadfbf34d [term 1 FOLLOWER]: Refusing update from remote peer 7133565a63d24fe09d46b0f58def254b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:37:10.991667 5527 consensus_queue.cc:1035] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P 7133565a63d24fe09d46b0f58def254b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:37:10.991786 5551 consensus_queue.cc:1035] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P 7133565a63d24fe09d46b0f58def254b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:37:11.002323 4526 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:37:11.002426 4526 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:37:11.003051 4526 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 028fd69b0c7645bca0e07c1476188e82: Sending DeleteTablet for 3 replicas of tablet 13c3aa3d51b145dd8f19c87f3dcc4a8f
I20260812 16:37:11.003392 4790 tablet_service.cc:1514] Processing DeleteTablet for tablet 13c3aa3d51b145dd8f19c87f3dcc4a8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:10 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:51386
I20260812 16:37:11.003422 5296 tablet_service.cc:1514] Processing DeleteTablet for tablet 13c3aa3d51b145dd8f19c87f3dcc4a8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:10 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:41812
I20260812 16:37:11.003499 5587 tablet_replica.cc:330] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P b36bcb39624f44849c3b9ccfadfbf34d: stopping tablet replica
I20260812 16:37:11.003515 5583 tablet_replica.cc:330] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P e1f11bd30d53435680b94197726c74bf: stopping tablet replica
I20260812 16:37:11.003500 5038 tablet_service.cc:1514] Processing DeleteTablet for tablet 13c3aa3d51b145dd8f19c87f3dcc4a8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:10 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:59048
I20260812 16:37:11.003553 5587 raft_consensus.cc:2229] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P b36bcb39624f44849c3b9ccfadfbf34d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:37:11.003564 5583 raft_consensus.cc:2229] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P e1f11bd30d53435680b94197726c74bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:37:11.003605 5587 raft_consensus.cc:2258] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P b36bcb39624f44849c3b9ccfadfbf34d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:11.003607 5583 raft_consensus.cc:2258] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P e1f11bd30d53435680b94197726c74bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:11.003613 5591 tablet_replica.cc:330] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P 7133565a63d24fe09d46b0f58def254b: stopping tablet replica
I20260812 16:37:11.003667 5591 raft_consensus.cc:2229] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P 7133565a63d24fe09d46b0f58def254b [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:37:11.003726 5591 raft_consensus.cc:2258] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P 7133565a63d24fe09d46b0f58def254b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:11.004040 5587 ts_tablet_manager.cc:1905] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P b36bcb39624f44849c3b9ccfadfbf34d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:37:11.004057 5583 ts_tablet_manager.cc:1905] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P e1f11bd30d53435680b94197726c74bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:37:11.004057 5591 ts_tablet_manager.cc:1905] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P 7133565a63d24fe09d46b0f58def254b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:37:11.005491 5587 ts_tablet_manager.cc:1918] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P b36bcb39624f44849c3b9ccfadfbf34d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:37:11.005580 5587 log.cc:1196] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P b36bcb39624f44849c3b9ccfadfbf34d: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-0/wal/wals/13c3aa3d51b145dd8f19c87f3dcc4a8f
I20260812 16:37:11.005736 5583 ts_tablet_manager.cc:1918] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P e1f11bd30d53435680b94197726c74bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:37:11.005815 5583 log.cc:1196] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P e1f11bd30d53435680b94197726c74bf: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-2/wal/wals/13c3aa3d51b145dd8f19c87f3dcc4a8f
I20260812 16:37:11.005824 5591 ts_tablet_manager.cc:1918] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P 7133565a63d24fe09d46b0f58def254b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:37:11.005860 5587 ts_tablet_manager.cc:1939] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P b36bcb39624f44849c3b9ccfadfbf34d: Deleting consensus metadata
I20260812 16:37:11.005879 5591 log.cc:1196] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P 7133565a63d24fe09d46b0f58def254b: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-1/wal/wals/13c3aa3d51b145dd8f19c87f3dcc4a8f
I20260812 16:37:11.006084 5583 ts_tablet_manager.cc:1939] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P e1f11bd30d53435680b94197726c74bf: Deleting consensus metadata
I20260812 16:37:11.006145 5591 ts_tablet_manager.cc:1939] T 13c3aa3d51b145dd8f19c87f3dcc4a8f P 7133565a63d24fe09d46b0f58def254b: Deleting consensus metadata
I20260812 16:37:11.006207 4496 catalog_manager.cc:4841] TS b36bcb39624f44849c3b9ccfadfbf34d (127.0.253.193:46443): tablet 13c3aa3d51b145dd8f19c87f3dcc4a8f (table test-table [id=4d03b298d4ea496c9a569f04b09aaf19]) successfully deleted
I20260812 16:37:11.006464 4496 catalog_manager.cc:4841] TS e1f11bd30d53435680b94197726c74bf (127.0.253.195:39525): tablet 13c3aa3d51b145dd8f19c87f3dcc4a8f (table test-table [id=4d03b298d4ea496c9a569f04b09aaf19]) successfully deleted
I20260812 16:37:11.006475 4497 catalog_manager.cc:4841] TS 7133565a63d24fe09d46b0f58def254b (127.0.253.194:37559): tablet 13c3aa3d51b145dd8f19c87f3dcc4a8f (table test-table [id=4d03b298d4ea496c9a569f04b09aaf19]) successfully deleted
I20260812 16:37:11.006510 4526 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38390:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:37:11.006806 4526 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:37:11.008329 4790 tablet_service.cc:1467] Processing CreateTablet for tablet af3b3c3b48e446bca47826e4fcefd4f9 (DEFAULT_TABLE table=test-table [id=367a71fc47d447b69625d724ee5899ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:11.008342 5296 tablet_service.cc:1467] Processing CreateTablet for tablet af3b3c3b48e446bca47826e4fcefd4f9 (DEFAULT_TABLE table=test-table [id=367a71fc47d447b69625d724ee5899ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:11.008456 5296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af3b3c3b48e446bca47826e4fcefd4f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:11.008456 4790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af3b3c3b48e446bca47826e4fcefd4f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:11.009788 5606 tablet_bootstrap.cc:492] T af3b3c3b48e446bca47826e4fcefd4f9 P b36bcb39624f44849c3b9ccfadfbf34d: Bootstrap starting.
I20260812 16:37:11.009858 5616 tablet_bootstrap.cc:492] T af3b3c3b48e446bca47826e4fcefd4f9 P e1f11bd30d53435680b94197726c74bf: Bootstrap starting.
I20260812 16:37:11.009894 5038 tablet_service.cc:1467] Processing CreateTablet for tablet af3b3c3b48e446bca47826e4fcefd4f9 (DEFAULT_TABLE table=test-table [id=367a71fc47d447b69625d724ee5899ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:11.010006 5038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af3b3c3b48e446bca47826e4fcefd4f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:11.010250 5616 tablet_bootstrap.cc:654] T af3b3c3b48e446bca47826e4fcefd4f9 P e1f11bd30d53435680b94197726c74bf: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:11.010291 5606 tablet_bootstrap.cc:654] T af3b3c3b48e446bca47826e4fcefd4f9 P b36bcb39624f44849c3b9ccfadfbf34d: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:11.010937 5616 tablet_bootstrap.cc:492] T af3b3c3b48e446bca47826e4fcefd4f9 P e1f11bd30d53435680b94197726c74bf: No bootstrap required, opened a new log
I20260812 16:37:11.011013 5616 ts_tablet_manager.cc:1397] T af3b3c3b48e446bca47826e4fcefd4f9 P e1f11bd30d53435680b94197726c74bf: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:11.011312 5607 tablet_bootstrap.cc:492] T af3b3c3b48e446bca47826e4fcefd4f9 P 7133565a63d24fe09d46b0f58def254b: Bootstrap starting.
I20260812 16:37:11.011248 5616 raft_consensus.cc:348] T af3b3c3b48e446bca47826e4fcefd4f9 P e1f11bd30d53435680b94197726c74bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:11.011355 5616 raft_consensus.cc:374] T af3b3c3b48e446bca47826e4fcefd4f9 P e1f11bd30d53435680b94197726c74bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:11.011386 5616 raft_consensus.cc:729] T af3b3c3b48e446bca47826e4fcefd4f9 P e1f11bd30d53435680b94197726c74bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f11bd30d53435680b94197726c74bf, State: Initialized, Role: FOLLOWER
I20260812 16:37:11.011461 5616 consensus_queue.cc:260] T af3b3c3b48e446bca47826e4fcefd4f9 P e1f11bd30d53435680b94197726c74bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:11.011579 5616 ts_tablet_manager.cc:1428] T af3b3c3b48e446bca47826e4fcefd4f9 P e1f11bd30d53435680b94197726c74bf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:11.011664 5607 tablet_bootstrap.cc:654] T af3b3c3b48e446bca47826e4fcefd4f9 P 7133565a63d24fe09d46b0f58def254b: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:11.011972 5606 tablet_bootstrap.cc:492] T af3b3c3b48e446bca47826e4fcefd4f9 P b36bcb39624f44849c3b9ccfadfbf34d: No bootstrap required, opened a new log
I20260812 16:37:11.012032 5606 ts_tablet_manager.cc:1397] T af3b3c3b48e446bca47826e4fcefd4f9 P b36bcb39624f44849c3b9ccfadfbf34d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:37:11.012207 5606 raft_consensus.cc:348] T af3b3c3b48e446bca47826e4fcefd4f9 P b36bcb39624f44849c3b9ccfadfbf34d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:11.012261 5606 raft_consensus.cc:374] T af3b3c3b48e446bca47826e4fcefd4f9 P b36bcb39624f44849c3b9ccfadfbf34d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:11.012276 5606 raft_consensus.cc:729] T af3b3c3b48e446bca47826e4fcefd4f9 P b36bcb39624f44849c3b9ccfadfbf34d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b36bcb39624f44849c3b9ccfadfbf34d, State: Initialized, Role: FOLLOWER
I20260812 16:37:11.012271 5607 tablet_bootstrap.cc:492] T af3b3c3b48e446bca47826e4fcefd4f9 P 7133565a63d24fe09d46b0f58def254b: No bootstrap required, opened a new log
I20260812 16:37:11.012316 5606 consensus_queue.cc:260] T af3b3c3b48e446bca47826e4fcefd4f9 P b36bcb39624f44849c3b9ccfadfbf34d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:11.012336 5607 ts_tablet_manager.cc:1397] T af3b3c3b48e446bca47826e4fcefd4f9 P 7133565a63d24fe09d46b0f58def254b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:11.012419 5606 ts_tablet_manager.cc:1428] T af3b3c3b48e446bca47826e4fcefd4f9 P b36bcb39624f44849c3b9ccfadfbf34d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:11.012445 5607 raft_consensus.cc:348] T af3b3c3b48e446bca47826e4fcefd4f9 P 7133565a63d24fe09d46b0f58def254b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:11.012518 5607 raft_consensus.cc:374] T af3b3c3b48e446bca47826e4fcefd4f9 P 7133565a63d24fe09d46b0f58def254b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:11.012547 5607 raft_consensus.cc:729] T af3b3c3b48e446bca47826e4fcefd4f9 P 7133565a63d24fe09d46b0f58def254b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7133565a63d24fe09d46b0f58def254b, State: Initialized, Role: FOLLOWER
I20260812 16:37:11.012620 5607 consensus_queue.cc:260] T af3b3c3b48e446bca47826e4fcefd4f9 P 7133565a63d24fe09d46b0f58def254b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:11.012729 5607 ts_tablet_manager.cc:1428] T af3b3c3b48e446bca47826e4fcefd4f9 P 7133565a63d24fe09d46b0f58def254b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:11.261365 5618 raft_consensus.cc:482] T af3b3c3b48e446bca47826e4fcefd4f9 P b36bcb39624f44849c3b9ccfadfbf34d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:37:11.261482 5618 raft_consensus.cc:504] T af3b3c3b48e446bca47826e4fcefd4f9 P b36bcb39624f44849c3b9ccfadfbf34d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:11.261953 5618 leader_election.cc:288] T af3b3c3b48e446bca47826e4fcefd4f9 P b36bcb39624f44849c3b9ccfadfbf34d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7133565a63d24fe09d46b0f58def254b (127.0.253.194:37559), e1f11bd30d53435680b94197726c74bf (127.0.253.195:39525)
I20260812 16:37:11.265234 5627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:11.262292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:36111 (local address 127.0.253.194:37559)
0812 16:37:11.262345 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:11.262347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:11.262408 (+ 61us) server_negotiation.cc:407] Connection header received
0812 16:37:11.262513 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:11.262514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:11.262521 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:11.262584 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:11.262821 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:11.263425 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:11.264450 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:11.264840 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:11.264933 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:11.264944 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:11.265109 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 16:37:11.265150 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 16:37:11.265241 5642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:11.262261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:45433 (local address 127.0.253.195:39525)
0812 16:37:11.262370 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:11.262373 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:11.262424 (+ 51us) server_negotiation.cc:407] Connection header received
0812 16:37:11.262517 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:11.262518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:11.262535 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:11.262684 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:11.262878 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:11.263454 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:11.264455 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:11.264853 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:11.264944 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:11.264952 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:11.265124 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 16:37:11.265162 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":51,"threads_started":1}
I20260812 16:37:11.265264 5640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:11.262135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:37559 (local address 127.0.253.193:36111)
0812 16:37:11.262348 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:11.262359 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:11.262428 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:11.262613 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:11.262614 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:11.262617 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:11.262722 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:11.262724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:11.263516 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:11.263518 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:11.264350 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:11.264352 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:11.264932 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:11.264933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:11.265017 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:11.265022 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:37:11.265098 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 16:37:11.265145 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":4,"thread_start_us":135,"threads_started":1}
I20260812 16:37:11.265266 5636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:11.262172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.195:39525 (local address 127.0.253.193:45433)
0812 16:37:11.262357 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:11.262370 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:11.262442 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:11.262695 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:11.262696 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:11.262698 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:11.262784 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:11.262785 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:11.263563 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:11.263564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:11.264353 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:11.264355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:11.264942 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:11.264943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:11.265034 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:11.265038 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:37:11.265103 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 16:37:11.265161 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":6,"thread_start_us":104,"threads_started":1}
I20260812 16:37:11.265544 5324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af3b3c3b48e446bca47826e4fcefd4f9" candidate_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f11bd30d53435680b94197726c74bf" is_pre_election: true
I20260812 16:37:11.265607 5075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af3b3c3b48e446bca47826e4fcefd4f9" candidate_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7133565a63d24fe09d46b0f58def254b" is_pre_election: true
I20260812 16:37:11.265673 5324 raft_consensus.cc:2454] T af3b3c3b48e446bca47826e4fcefd4f9 P e1f11bd30d53435680b94197726c74bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b36bcb39624f44849c3b9ccfadfbf34d in term 0.
I20260812 16:37:11.265720 5075 raft_consensus.cc:2454] T af3b3c3b48e446bca47826e4fcefd4f9 P 7133565a63d24fe09d46b0f58def254b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b36bcb39624f44849c3b9ccfadfbf34d in term 0.
I20260812 16:37:11.265853 4694 leader_election.cc:302] T af3b3c3b48e446bca47826e4fcefd4f9 P b36bcb39624f44849c3b9ccfadfbf34d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b36bcb39624f44849c3b9ccfadfbf34d, e1f11bd30d53435680b94197726c74bf; no voters:
I20260812 16:37:11.266038 5618 raft_consensus.cc:2781] T af3b3c3b48e446bca47826e4fcefd4f9 P b36bcb39624f44849c3b9ccfadfbf34d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:37:11.266076 5618 raft_consensus.cc:482] T af3b3c3b48e446bca47826e4fcefd4f9 P b36bcb39624f44849c3b9ccfadfbf34d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:37:11.266091 5618 raft_consensus.cc:3037] T af3b3c3b48e446bca47826e4fcefd4f9 P b36bcb39624f44849c3b9ccfadfbf34d [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:11.266798 5618 raft_consensus.cc:504] T af3b3c3b48e446bca47826e4fcefd4f9 P b36bcb39624f44849c3b9ccfadfbf34d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:11.266954 5618 leader_election.cc:288] T af3b3c3b48e446bca47826e4fcefd4f9 P b36bcb39624f44849c3b9ccfadfbf34d [CANDIDATE]: Term 1 election: Requested vote from peers 7133565a63d24fe09d46b0f58def254b (127.0.253.194:37559), e1f11bd30d53435680b94197726c74bf (127.0.253.195:39525)
I20260812 16:37:11.267161 5324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af3b3c3b48e446bca47826e4fcefd4f9" candidate_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f11bd30d53435680b94197726c74bf"
I20260812 16:37:11.267148 5075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af3b3c3b48e446bca47826e4fcefd4f9" candidate_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7133565a63d24fe09d46b0f58def254b"
I20260812 16:37:11.267231 5324 raft_consensus.cc:3037] T af3b3c3b48e446bca47826e4fcefd4f9 P e1f11bd30d53435680b94197726c74bf [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:11.267251 5075 raft_consensus.cc:3037] T af3b3c3b48e446bca47826e4fcefd4f9 P 7133565a63d24fe09d46b0f58def254b [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:11.267908 5324 raft_consensus.cc:2454] T af3b3c3b48e446bca47826e4fcefd4f9 P e1f11bd30d53435680b94197726c74bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b36bcb39624f44849c3b9ccfadfbf34d in term 1.
I20260812 16:37:11.267998 5075 raft_consensus.cc:2454] T af3b3c3b48e446bca47826e4fcefd4f9 P 7133565a63d24fe09d46b0f58def254b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b36bcb39624f44849c3b9ccfadfbf34d in term 1.
I20260812 16:37:11.268142 4694 leader_election.cc:302] T af3b3c3b48e446bca47826e4fcefd4f9 P b36bcb39624f44849c3b9ccfadfbf34d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b36bcb39624f44849c3b9ccfadfbf34d, e1f11bd30d53435680b94197726c74bf; no voters:
I20260812 16:37:11.268251 5618 raft_consensus.cc:2781] T af3b3c3b48e446bca47826e4fcefd4f9 P b36bcb39624f44849c3b9ccfadfbf34d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:37:11.268437 5618 raft_consensus.cc:686] T af3b3c3b48e446bca47826e4fcefd4f9 P b36bcb39624f44849c3b9ccfadfbf34d [term 1 LEADER]: Becoming Leader. State: Replica: b36bcb39624f44849c3b9ccfadfbf34d, State: Running, Role: LEADER
I20260812 16:37:11.268530 5618 consensus_queue.cc:237] T af3b3c3b48e446bca47826e4fcefd4f9 P b36bcb39624f44849c3b9ccfadfbf34d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } }
I20260812 16:37:11.269384 4526 catalog_manager.cc:5495] T af3b3c3b48e446bca47826e4fcefd4f9 P b36bcb39624f44849c3b9ccfadfbf34d reported cstate change: term changed from 0 to 1, leader changed from <none> to b36bcb39624f44849c3b9ccfadfbf34d (127.0.253.193). New cstate: current_term: 1 leader_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36bcb39624f44849c3b9ccfadfbf34d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 46443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 } health_report { overall_health: UNKNOWN } } }
I20260812 16:37:11.298127 5646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:11.295545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51394 (local address 127.0.253.193:46443)
0812 16:37:11.295763 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:11.295765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:11.295773 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:37:11.295812 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:11.295812 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:11.295833 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:11.295908 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:37:11.296041 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:11.296543 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:11.297318 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:11.297524 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:11.297580 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:11.297750 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:37:11.297875 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:37:11.297933 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:11.297982 (+ 49us) server_negotiation.cc:299] Negotiation successful
0812 16:37:11.297999 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":142,"threads_started":1}
I20260812 16:37:11.303627 5627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:11.301323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:50011 (local address 127.0.253.194:37559)
0812 16:37:11.301397 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:11.301399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:11.301553 (+ 154us) server_negotiation.cc:407] Connection header received
0812 16:37:11.301589 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:11.301590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:11.301599 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:11.301644 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:37:11.301747 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:11.302214 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:11.302980 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:11.303198 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:11.303255 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:11.303429 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:37:11.303459 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:37:11.303497 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:11.303551 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 16:37:11.303569 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 16:37:11.303686 5651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:11.301253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:37559 (local address 127.0.253.193:50011)
0812 16:37:11.301481 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:11.301492 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:11.301548 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:11.301652 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:11.301653 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:11.301654 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 16:37:11.301709 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:11.301710 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:11.302285 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:11.302286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:11.302887 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:11.302889 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:11.303289 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:11.303290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:11.303341 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:11.303355 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 16:37:11.303514 (+ 159us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 16:37:11.303515 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 16:37:11.303550 (+ 35us) client_negotiation.cc:240] Negotiation successful
0812 16:37:11.303616 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":116,"threads_started":1}
I20260812 16:37:11.305969 5324 raft_consensus.cc:1261] T af3b3c3b48e446bca47826e4fcefd4f9 P e1f11bd30d53435680b94197726c74bf [term 1 FOLLOWER]: Refusing update from remote peer b36bcb39624f44849c3b9ccfadfbf34d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:37:11.306281 5075 raft_consensus.cc:1261] T af3b3c3b48e446bca47826e4fcefd4f9 P 7133565a63d24fe09d46b0f58def254b [term 1 FOLLOWER]: Refusing update from remote peer b36bcb39624f44849c3b9ccfadfbf34d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:37:11.306296 5643 consensus_queue.cc:1035] T af3b3c3b48e446bca47826e4fcefd4f9 P b36bcb39624f44849c3b9ccfadfbf34d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1f11bd30d53435680b94197726c74bf" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 39525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:37:11.306479 5643 consensus_queue.cc:1035] T af3b3c3b48e446bca47826e4fcefd4f9 P b36bcb39624f44849c3b9ccfadfbf34d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7133565a63d24fe09d46b0f58def254b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:37:11.314316 5646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:11.311733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:41471 (local address 127.0.253.193:46443)
0812 16:37:11.311779 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:11.311782 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:11.311929 (+ 147us) server_negotiation.cc:407] Connection header received
0812 16:37:11.311955 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:11.311956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:11.311968 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:11.312022 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:37:11.312208 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:11.312742 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:11.313600 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:11.313912 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:11.313977 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:11.314124 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:37:11.314150 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:37:11.314193 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:11.314249 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 16:37:11.314265 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 16:37:11.314365 5659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:11.311656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:46443 (local address 127.0.253.194:41471)
0812 16:37:11.311817 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:11.311829 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:11.311891 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:11.312059 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:11.312059 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:11.312061 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 16:37:11.312134 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:11.312136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:11.312826 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:11.312827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:11.313503 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:11.313506 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:11.313987 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:11.313988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:11.314047 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:11.314053 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 16:37:11.314196 (+ 143us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 16:37:11.314198 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 16:37:11.314248 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 16:37:11.314298 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":109,"threads_started":1}
W20260812 16:37:11.314706 5560 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 16:37:12.839840 4525 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:37:12.839956 4525 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:37:12.840891 4525 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 028fd69b0c7645bca0e07c1476188e82: Sending DeleteTablet for 3 replicas of tablet af3b3c3b48e446bca47826e4fcefd4f9
I20260812 16:37:12.841171 5296 tablet_service.cc:1514] Processing DeleteTablet for tablet af3b3c3b48e446bca47826e4fcefd4f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:12 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:41812
I20260812 16:37:12.841328 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 4662
I20260812 16:37:12.841375 4790 tablet_service.cc:1514] Processing DeleteTablet for tablet af3b3c3b48e446bca47826e4fcefd4f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:12 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:51386
I20260812 16:37:12.841393 5701 tablet_replica.cc:330] T af3b3c3b48e446bca47826e4fcefd4f9 P e1f11bd30d53435680b94197726c74bf: stopping tablet replica
I20260812 16:37:12.841461 5701 raft_consensus.cc:2229] T af3b3c3b48e446bca47826e4fcefd4f9 P e1f11bd30d53435680b94197726c74bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:37:12.841498 5701 raft_consensus.cc:2258] T af3b3c3b48e446bca47826e4fcefd4f9 P e1f11bd30d53435680b94197726c74bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:12.841862 5038 tablet_service.cc:1514] Processing DeleteTablet for tablet af3b3c3b48e446bca47826e4fcefd4f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:12 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:59048
I20260812 16:37:12.842183 5702 tablet_replica.cc:330] T af3b3c3b48e446bca47826e4fcefd4f9 P 7133565a63d24fe09d46b0f58def254b: stopping tablet replica
I20260812 16:37:12.842271 5702 raft_consensus.cc:2229] T af3b3c3b48e446bca47826e4fcefd4f9 P 7133565a63d24fe09d46b0f58def254b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:37:12.842339 5702 raft_consensus.cc:2258] T af3b3c3b48e446bca47826e4fcefd4f9 P 7133565a63d24fe09d46b0f58def254b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:12.842585 5702 ts_tablet_manager.cc:1905] T af3b3c3b48e446bca47826e4fcefd4f9 P 7133565a63d24fe09d46b0f58def254b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:37:12.844811 5702 ts_tablet_manager.cc:1918] T af3b3c3b48e446bca47826e4fcefd4f9 P 7133565a63d24fe09d46b0f58def254b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 16:37:12.844877 5702 log.cc:1196] T af3b3c3b48e446bca47826e4fcefd4f9 P 7133565a63d24fe09d46b0f58def254b: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-1/wal/wals/af3b3c3b48e446bca47826e4fcefd4f9
I20260812 16:37:12.845077 5702 ts_tablet_manager.cc:1939] T af3b3c3b48e446bca47826e4fcefd4f9 P 7133565a63d24fe09d46b0f58def254b: Deleting consensus metadata
I20260812 16:37:12.847200 4497 catalog_manager.cc:4841] TS 7133565a63d24fe09d46b0f58def254b (127.0.253.194:37559): tablet af3b3c3b48e446bca47826e4fcefd4f9 (table test-table [id=367a71fc47d447b69625d724ee5899ac]) successfully deleted
I20260812 16:37:12.848860 5701 ts_tablet_manager.cc:1905] T af3b3c3b48e446bca47826e4fcefd4f9 P e1f11bd30d53435680b94197726c74bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:37:12.851115 5701 ts_tablet_manager.cc:1918] T af3b3c3b48e446bca47826e4fcefd4f9 P e1f11bd30d53435680b94197726c74bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 16:37:12.851186 5701 log.cc:1196] T af3b3c3b48e446bca47826e4fcefd4f9 P e1f11bd30d53435680b94197726c74bf: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552619305671-1015-0/minicluster-data/ts-2/wal/wals/af3b3c3b48e446bca47826e4fcefd4f9
I20260812 16:37:12.851382 5701 ts_tablet_manager.cc:1939] T af3b3c3b48e446bca47826e4fcefd4f9 P e1f11bd30d53435680b94197726c74bf: Deleting consensus metadata
I20260812 16:37:12.851684 4496 catalog_manager.cc:4841] TS e1f11bd30d53435680b94197726c74bf (127.0.253.195:39525): tablet af3b3c3b48e446bca47826e4fcefd4f9 (table test-table [id=367a71fc47d447b69625d724ee5899ac]) successfully deleted
W20260812 16:37:12.861388 4496 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.253.193:46443 (error 108)
I20260812 16:37:12.861691 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 4927
I20260812 16:37:12.873519 5707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:12.871068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:46443 (local address 127.0.0.1:51404)
0812 16:37:12.873383 (+ 2315us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:12.873468 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.193:46443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2251,"thread_start_us":113,"threads_started":1}
W20260812 16:37:12.873744 4496 catalog_manager.cc:4584] TS b36bcb39624f44849c3b9ccfadfbf34d (127.0.253.193:46443): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet af3b3c3b48e446bca47826e4fcefd4f9: Network error: Client connection negotiation failed: client connection to 127.0.253.193:46443: connect: Connection refused (error 111)
I20260812 16:37:12.879550 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 5192
I20260812 16:37:12.895641 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 4466
2026-08-12T16:37:12Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6321 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:37:12 dist-test-slave-w4v5 krb5kdc[5713](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:37:12 dist-test-slave-w4v5 krb5kdc[5713](info): set up 2 sockets
Aug 12 16:37:12 dist-test-slave-w4v5 krb5kdc[5713](info): commencing operation
krb5kdc: starting...
W20260812 16:37:14.217375 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.288s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:37:14 dist-test-slave-w4v5 krb5kdc[5713](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:37:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:37:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
I20260812 16:37:14.346275 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:37471
--webserver_interface=127.0.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:41423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:37471
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:14.429575 5745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:14.429821 5745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:14.429857 5745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:14.431007 5745 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:37:14.431085 5745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:14.431147 5745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:14.431176 5745 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:37:14.431207 5745 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:37:14.432708 5745 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:41423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:37471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:37471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:14.432994 5745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:14.433238 5745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 16:37:14.438578 5745 server_base.cc:1034] running on GCE node
W20260812 16:37:14.438728 5756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:14.438642 5754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:14.438634 5758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:14.439139 5745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:14.439447 5745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:14.440632 5745 hybrid_clock.cc:648] HybridClock initialized: now 1786552634440612 us; error 33 us; skew 500 ppm
Aug 12 16:37:14 dist-test-slave-w4v5 krb5kdc[5713](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552634, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:14.449048 5745 init.cc:377] Logged in from keytab as kudu/127.0.253.254@KRBTEST.COM (short username kudu)
I20260812 16:37:14.449569 5745 webserver.cc:460] Webserver started at http://127.0.253.254:43571/ using document root <none> and password file <none>
I20260812 16:37:14.449823 5745 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:14.449894 5745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:14.450043 5745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:14.451330 5745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "1e9a493a196d4f15bce6da0ddc61d169"
format_stamp: "Formatted at 2026-08-12 16:37:14 on dist-test-slave-w4v5"
server_key: "f08271dc2d08007a58564fd3cc8440d4"
server_key_iv: "81b77d21486804f709f19578a38d439a"
server_key_version: "encryptionkey@0"
I20260812 16:37:14.451727 5745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "1e9a493a196d4f15bce6da0ddc61d169"
format_stamp: "Formatted at 2026-08-12 16:37:14 on dist-test-slave-w4v5"
server_key: "f08271dc2d08007a58564fd3cc8440d4"
server_key_iv: "81b77d21486804f709f19578a38d439a"
server_key_version: "encryptionkey@0"
I20260812 16:37:14.453110 5745 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:37:14.453862 5767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:14.454073 5745 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:14.454174 5745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "1e9a493a196d4f15bce6da0ddc61d169"
format_stamp: "Formatted at 2026-08-12 16:37:14 on dist-test-slave-w4v5"
server_key: "f08271dc2d08007a58564fd3cc8440d4"
server_key_iv: "81b77d21486804f709f19578a38d439a"
server_key_version: "encryptionkey@0"
I20260812 16:37:14.454233 5745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:14.462970 5745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:14.464973 5745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:14.465085 5745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:14.469061 5745 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:37471
I20260812 16:37:14.469115 5869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:37471 every 8 connection(s)
I20260812 16:37:14.469447 5745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:37:14.470448 5871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:14.471575 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 5745
I20260812 16:37:14.471673 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:37:14.471853 1015 external_mini_cluster.cc:1428] Setting key daa85bf607222a50727c65f9e6ae6afe
I20260812 16:37:14.474079 5871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e9a493a196d4f15bce6da0ddc61d169: Bootstrap starting.
I20260812 16:37:14.474691 5871 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e9a493a196d4f15bce6da0ddc61d169: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:14.475000 5871 log.cc:824] T 00000000000000000000000000000000 P 1e9a493a196d4f15bce6da0ddc61d169: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:14.475839 5871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e9a493a196d4f15bce6da0ddc61d169: No bootstrap required, opened a new log
I20260812 16:37:14.478458 5871 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e9a493a196d4f15bce6da0ddc61d169 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e9a493a196d4f15bce6da0ddc61d169" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 37471 } }
I20260812 16:37:14.478608 5871 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e9a493a196d4f15bce6da0ddc61d169 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:14.478633 5871 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e9a493a196d4f15bce6da0ddc61d169 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e9a493a196d4f15bce6da0ddc61d169, State: Initialized, Role: FOLLOWER
I20260812 16:37:14.478772 5871 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e9a493a196d4f15bce6da0ddc61d169 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e9a493a196d4f15bce6da0ddc61d169" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 37471 } }
I20260812 16:37:14.478852 5871 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e9a493a196d4f15bce6da0ddc61d169 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:37:14.478876 5871 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e9a493a196d4f15bce6da0ddc61d169 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
Aug 12 16:37:14 dist-test-slave-w4v5 krb5kdc[5713](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:14.478943 5871 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e9a493a196d4f15bce6da0ddc61d169 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:14.479641 5871 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e9a493a196d4f15bce6da0ddc61d169 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e9a493a196d4f15bce6da0ddc61d169" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 37471 } }
I20260812 16:37:14.479765 5871 leader_election.cc:302] T 00000000000000000000000000000000 P 1e9a493a196d4f15bce6da0ddc61d169 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e9a493a196d4f15bce6da0ddc61d169; no voters:
I20260812 16:37:14.479957 5871 leader_election.cc:288] T 00000000000000000000000000000000 P 1e9a493a196d4f15bce6da0ddc61d169 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:37:14.479990 5876 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e9a493a196d4f15bce6da0ddc61d169 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:37:14.480119 5876 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e9a493a196d4f15bce6da0ddc61d169 [term 1 LEADER]: Becoming Leader. State: Replica: 1e9a493a196d4f15bce6da0ddc61d169, State: Running, Role: LEADER
I20260812 16:37:14.480243 5876 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e9a493a196d4f15bce6da0ddc61d169 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e9a493a196d4f15bce6da0ddc61d169" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 37471 } }
I20260812 16:37:14.480250 5871 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e9a493a196d4f15bce6da0ddc61d169 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:37:14.480670 5882 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e9a493a196d4f15bce6da0ddc61d169 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e9a493a196d4f15bce6da0ddc61d169. Latest consensus state: current_term: 1 leader_uuid: "1e9a493a196d4f15bce6da0ddc61d169" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e9a493a196d4f15bce6da0ddc61d169" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 37471 } } }
I20260812 16:37:14.480664 5879 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e9a493a196d4f15bce6da0ddc61d169 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e9a493a196d4f15bce6da0ddc61d169" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e9a493a196d4f15bce6da0ddc61d169" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 37471 } } }
I20260812 16:37:14.480834 5879 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e9a493a196d4f15bce6da0ddc61d169 [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:14.481014 5882 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e9a493a196d4f15bce6da0ddc61d169 [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:14.481196 5886 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:37:14.481791 5886 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:37:14.483520 5886 catalog_manager.cc:1347] Generated new cluster ID: a3e77f7ae52b4d43a0996862537766cc
I20260812 16:37:14.483574 5886 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:37:14.484144 5874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:14.472379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50652 (local address 127.0.253.254:37471)
0812 16:37:14.472560 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:14.472568 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:14.472592 (+ 24us) server_negotiation.cc:407] Connection header received
0812 16:37:14.473021 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:14.473029 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:14.473075 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:14.473188 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:14.474003 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:14.474992 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:14.476048 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:14.476454 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:14.476538 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:14.479479 (+ 2941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:14.479484 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:14.479485 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:14.479499 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:14.482062 (+ 2563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:14.482580 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:14.482583 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:14.482583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:14.482687 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:14.482937 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:14.482938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:14.482938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:14.483272 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:14.483423 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:14.483792 (+ 369us) server_negotiation.cc:299] Negotiation successful
0812 16:37:14.483932 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":91,"threads_started":1}
I20260812 16:37:14.488901 5886 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:37:14.489674 5886 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:37:14.505208 5886 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e9a493a196d4f15bce6da0ddc61d169: Generated new TSK 0
I20260812 16:37:14.505452 5886 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab.
Entry for principal kudu/127.0.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab.
I20260812 16:37:14.538189 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.193:0
--local_ip_for_outbound_sockets=127.0.253.193
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:37471
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:41423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:14.625226 5922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:14.625475 5922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:14.625519 5922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:14.626706 5922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:14.626776 5922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:14.626822 5922 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.193
I20260812 16:37:14.628422 5922 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:41423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:37471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.193
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:14.628705 5922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:14.628952 5922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:14.634368 5931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:14.634368 5934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:14.634388 5930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:14.634397 5922 server_base.cc:1034] running on GCE node
I20260812 16:37:14.635010 5922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:14.635301 5922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:14.636482 5922 hybrid_clock.cc:648] HybridClock initialized: now 1786552634636470 us; error 43 us; skew 500 ppm
Aug 12 16:37:14 dist-test-slave-w4v5 krb5kdc[5713](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552634, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:14.639369 5922 init.cc:377] Logged in from keytab as kudu/127.0.253.193@KRBTEST.COM (short username kudu)
I20260812 16:37:14.639863 5922 webserver.cc:460] Webserver started at http://127.0.253.193:36921/ using document root <none> and password file <none>
I20260812 16:37:14.640132 5922 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:14.640208 5922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:14.640355 5922 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:14.641464 5922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-0/data/instance:
uuid: "2fdfa5fd29e24922ba7d7b9873df947f"
format_stamp: "Formatted at 2026-08-12 16:37:14 on dist-test-slave-w4v5"
server_key: "9b31a33ed9fba874e5d533875d95b510"
server_key_iv: "f201549b181cfa3f04efd59afd2f33b5"
server_key_version: "encryptionkey@0"
I20260812 16:37:14.641842 5922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance:
uuid: "2fdfa5fd29e24922ba7d7b9873df947f"
format_stamp: "Formatted at 2026-08-12 16:37:14 on dist-test-slave-w4v5"
server_key: "9b31a33ed9fba874e5d533875d95b510"
server_key_iv: "f201549b181cfa3f04efd59afd2f33b5"
server_key_version: "encryptionkey@0"
I20260812 16:37:14.643319 5922 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:37:14.644069 5945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:14.644308 5922 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:14.644423 5922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-0/wal
uuid: "2fdfa5fd29e24922ba7d7b9873df947f"
format_stamp: "Formatted at 2026-08-12 16:37:14 on dist-test-slave-w4v5"
server_key: "9b31a33ed9fba874e5d533875d95b510"
server_key_iv: "f201549b181cfa3f04efd59afd2f33b5"
server_key_version: "encryptionkey@0"
I20260812 16:37:14.644495 5922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:14.656706 5922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:14.659004 5922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:14.659220 5922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:14.659471 5922 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:37:14.659796 5922 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:14.659832 5922 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:14.659876 5922 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:14.659902 5922 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:14.665871 5922 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.193:44841
I20260812 16:37:14.665951 6151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.193:44841 every 8 connection(s)
I20260812 16:37:14.666293 5922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
Aug 12 16:37:14 dist-test-slave-w4v5 krb5kdc[5713](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552634, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:14.674850 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 5922
I20260812 16:37:14.674950 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance
I20260812 16:37:14.675195 1015 external_mini_cluster.cc:1428] Setting key b11b8914f3d1825ecfff19ad77bf9f3a
I20260812 16:37:14.675875 6158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:14.667033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:37471 (local address 127.0.253.193:37855)
0812 16:37:14.667241 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:14.667285 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:14.667769 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:14.667982 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:14.667985 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:14.668577 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:14.668774 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:14.668779 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:14.669440 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:14.669441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:14.670245 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:14.670248 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:14.670631 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:14.670632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:14.670741 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:14.671264 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:14.671272 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:14.672652 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:14.674317 (+ 1665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:14.674323 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:14.674324 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:14.674526 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:14.674719 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:14.674720 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:14.674721 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:14.674866 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:14.675268 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:14.675277 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:14.675443 (+ 166us) client_negotiation.cc:769] Sending connection context
0812 16:37:14.675572 (+ 129us) client_negotiation.cc:240] Negotiation successful
0812 16:37:14.675717 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":139,"threads_started":1}
I20260812 16:37:14.675910 5874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:14.667124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:37855 (local address 127.0.253.254:37471)
0812 16:37:14.667177 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:14.667179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:14.667380 (+ 201us) server_negotiation.cc:407] Connection header received
0812 16:37:14.667857 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:14.667858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:14.667868 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:14.667928 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:14.668861 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:14.669355 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:14.670334 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:14.670553 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:14.670615 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:14.672760 (+ 2145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:14.672762 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:14.672762 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:14.672769 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:14.674248 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:14.674591 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:14.674592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:14.674593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:14.674650 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:14.674995 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:14.674996 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:14.674997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:14.675178 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:14.675252 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:14.675742 (+ 490us) server_negotiation.cc:299] Negotiation successful
0812 16:37:14.675829 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 16:37:14.676499 6152 heartbeater.cc:347] Connected to a master server at 127.0.253.254:37471
I20260812 16:37:14.676606 6152 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:14.676785 6152 heartbeater.cc:511] Master 127.0.253.254:37471 requested a full tablet report, sending...
I20260812 16:37:14.677366 5806 ts_manager.cc:194] Registered new tserver with Master: 2fdfa5fd29e24922ba7d7b9873df947f (127.0.253.193:44841)
I20260812 16:37:14.678277 5806 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.193@KRBTEST.COM'} at 127.0.253.193:37855
WARNING: no policy specified for kudu/127.0.253.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab.
Entry for principal kudu/127.0.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab.
I20260812 16:37:14.702468 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.194:0
--local_ip_for_outbound_sockets=127.0.253.194
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:37471
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:41423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:14.789737 6163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:14.789986 6163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:14.790025 6163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:14.791234 6163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:14.791303 6163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:14.791347 6163 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.194
I20260812 16:37:14.792830 6163 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:41423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:37471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.194
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:14.793109 6163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:14.793356 6163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 16:37:14.798964 6163 server_base.cc:1034] running on GCE node
W20260812 16:37:14.798875 6171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:14.798837 6169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:14.798851 6173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:14.799345 6163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:14.799610 6163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:14.800761 6163 hybrid_clock.cc:648] HybridClock initialized: now 1786552634800764 us; error 44 us; skew 500 ppm
Aug 12 16:37:14 dist-test-slave-w4v5 krb5kdc[5713](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552634, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:14.803750 6163 init.cc:377] Logged in from keytab as kudu/127.0.253.194@KRBTEST.COM (short username kudu)
I20260812 16:37:14.804344 6163 webserver.cc:460] Webserver started at http://127.0.253.194:35159/ using document root <none> and password file <none>
I20260812 16:37:14.804639 6163 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:14.804714 6163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:14.804863 6163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:14.806344 6163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-1/data/instance:
uuid: "4f776a1d7c0442bc850927725744d9f2"
format_stamp: "Formatted at 2026-08-12 16:37:14 on dist-test-slave-w4v5"
server_key: "1b0de7fa7ea5958cc3c36cb52bc16464"
server_key_iv: "b203e515c73713d90b19de9eed16dea7"
server_key_version: "encryptionkey@0"
I20260812 16:37:14.806710 6163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance:
uuid: "4f776a1d7c0442bc850927725744d9f2"
format_stamp: "Formatted at 2026-08-12 16:37:14 on dist-test-slave-w4v5"
server_key: "1b0de7fa7ea5958cc3c36cb52bc16464"
server_key_iv: "b203e515c73713d90b19de9eed16dea7"
server_key_version: "encryptionkey@0"
I20260812 16:37:14.808287 6163 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260812 16:37:14.809082 6182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:14.809252 6163 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:37:14.809355 6163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-1/wal
uuid: "4f776a1d7c0442bc850927725744d9f2"
format_stamp: "Formatted at 2026-08-12 16:37:14 on dist-test-slave-w4v5"
server_key: "1b0de7fa7ea5958cc3c36cb52bc16464"
server_key_iv: "b203e515c73713d90b19de9eed16dea7"
server_key_version: "encryptionkey@0"
I20260812 16:37:14.809440 6163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:14.823190 6163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:14.825677 6163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:14.825848 6163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:14.826122 6163 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:37:14.826452 6163 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:14.826508 6163 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:14.826551 6163 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:14.826581 6163 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:14.832955 6163 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.194:45993
I20260812 16:37:14.833009 6378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.194:45993 every 8 connection(s)
I20260812 16:37:14.833348 6163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
I20260812 16:37:14.837980 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 6163
I20260812 16:37:14.838084 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance
I20260812 16:37:14.838290 1015 external_mini_cluster.cc:1428] Setting key 3127cdd0548fbfa6e9e9469f01eb4e4e
Aug 12 16:37:14 dist-test-slave-w4v5 krb5kdc[5713](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552634, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:14.844173 5874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:14.834142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:38665 (local address 127.0.253.254:37471)
0812 16:37:14.834217 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:14.834219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:14.834457 (+ 238us) server_negotiation.cc:407] Connection header received
0812 16:37:14.834921 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:14.834922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:14.834933 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:14.834990 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:14.836040 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:14.836536 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:14.837582 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:14.837848 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:14.837937 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:14.840851 (+ 2914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:14.840853 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:14.840853 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:14.840861 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:14.842573 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:14.842976 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:14.842978 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:14.842978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:14.843048 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:14.843383 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:14.843384 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:14.843385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:14.843548 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:14.843622 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:14.843959 (+ 337us) server_negotiation.cc:299] Negotiation successful
0812 16:37:14.844059 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 16:37:14.844347 6381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:14.834124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:37471 (local address 127.0.253.194:38665)
0812 16:37:14.834306 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:14.834344 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:14.834839 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:14.835118 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:14.835123 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:14.835737 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:14.835938 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:14.835944 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:14.836643 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:14.836644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:14.837467 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:14.837469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:14.837940 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:14.837941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:14.838079 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:14.838743 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:14.838754 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:14.840741 (+ 1987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:14.842663 (+ 1922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:14.842670 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:14.842672 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:14.842897 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:14.843192 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:14.843193 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:14.843194 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:14.843308 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:14.843640 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:14.843647 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:14.843776 (+ 129us) client_negotiation.cc:769] Sending connection context
0812 16:37:14.843882 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 16:37:14.844018 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":88,"threads_started":1}
I20260812 16:37:14.844913 6379 heartbeater.cc:347] Connected to a master server at 127.0.253.254:37471
I20260812 16:37:14.845023 6379 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:14.845206 6379 heartbeater.cc:511] Master 127.0.253.254:37471 requested a full tablet report, sending...
I20260812 16:37:14.845681 5806 ts_manager.cc:194] Registered new tserver with Master: 4f776a1d7c0442bc850927725744d9f2 (127.0.253.194:45993)
I20260812 16:37:14.846259 5806 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.194@KRBTEST.COM'} at 127.0.253.194:38665
WARNING: no policy specified for kudu/127.0.253.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab.
Entry for principal kudu/127.0.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab.
I20260812 16:37:14.867908 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.195:0
--local_ip_for_outbound_sockets=127.0.253.195
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:37471
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:41423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:14.950371 6385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:14.950593 6385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:14.950618 6385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:14.951915 6385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:14.951956 6385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:14.951983 6385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.195
I20260812 16:37:14.953380 6385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:41423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:37471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.195
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:14.953569 6385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:14.953733 6385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:14.958920 6397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:14.958962 6393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:14.958962 6394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:14.959296 6385 server_base.cc:1034] running on GCE node
I20260812 16:37:14.959540 6385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:14.959777 6385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:14.960951 6385 hybrid_clock.cc:648] HybridClock initialized: now 1786552634960930 us; error 31 us; skew 500 ppm
Aug 12 16:37:14 dist-test-slave-w4v5 krb5kdc[5713](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552634, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:14.963694 6385 init.cc:377] Logged in from keytab as kudu/127.0.253.195@KRBTEST.COM (short username kudu)
I20260812 16:37:14.964102 6385 webserver.cc:460] Webserver started at http://127.0.253.195:40553/ using document root <none> and password file <none>
I20260812 16:37:14.964301 6385 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:14.964342 6385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:14.964427 6385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:14.965440 6385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-2/data/instance:
uuid: "174d1019402c4ba8a63147f8c946b8d6"
format_stamp: "Formatted at 2026-08-12 16:37:14 on dist-test-slave-w4v5"
server_key: "47b33c267dafd5928dd7f84f9f401b93"
server_key_iv: "908ef94a6aedeca432fcb120d298f6ef"
server_key_version: "encryptionkey@0"
I20260812 16:37:14.965737 6385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance:
uuid: "174d1019402c4ba8a63147f8c946b8d6"
format_stamp: "Formatted at 2026-08-12 16:37:14 on dist-test-slave-w4v5"
server_key: "47b33c267dafd5928dd7f84f9f401b93"
server_key_iv: "908ef94a6aedeca432fcb120d298f6ef"
server_key_version: "encryptionkey@0"
I20260812 16:37:14.967029 6385 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 16:37:14.967840 6406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:14.968009 6385 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 16:37:14.968132 6385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-2/wal
uuid: "174d1019402c4ba8a63147f8c946b8d6"
format_stamp: "Formatted at 2026-08-12 16:37:14 on dist-test-slave-w4v5"
server_key: "47b33c267dafd5928dd7f84f9f401b93"
server_key_iv: "908ef94a6aedeca432fcb120d298f6ef"
server_key_version: "encryptionkey@0"
I20260812 16:37:14.968207 6385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:14.978822 6385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:14.981194 6385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:14.981357 6385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:14.981578 6385 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:37:14.981935 6385 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:14.981990 6385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:14.982034 6385 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:14.982062 6385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:14.988255 6385 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.195:40895
I20260812 16:37:14.988310 6604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.195:40895 every 8 connection(s)
I20260812 16:37:14.988633 6385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
I20260812 16:37:14.993563 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 6385
I20260812 16:37:14.993671 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance
I20260812 16:37:14.993867 1015 external_mini_cluster.cc:1428] Setting key 6d99160c5785ffb8a7fdd265b56a31b9
Aug 12 16:37:14 dist-test-slave-w4v5 krb5kdc[5713](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552634, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:14.998432 5874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:14.989615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:49487 (local address 127.0.253.254:37471)
0812 16:37:14.989675 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:14.989677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:14.989883 (+ 206us) server_negotiation.cc:407] Connection header received
0812 16:37:14.990328 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:14.990329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:14.990339 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:14.990392 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:14.991469 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:14.991979 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:14.993054 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:14.993250 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:14.993309 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:14.995979 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:14.995981 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:14.995981 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:14.995997 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:14.997183 (+ 1186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:14.997570 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:14.997571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:14.997571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:14.997632 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:14.997866 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:14.997866 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:14.997867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:14.997982 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:14.998050 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:14.998294 (+ 244us) server_negotiation.cc:299] Negotiation successful
0812 16:37:14.998367 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 16:37:14.998570 6610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:14.989530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:37471 (local address 127.0.253.195:49487)
0812 16:37:14.989762 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:14.989798 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:14.990247 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:14.990446 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:14.990450 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:14.991122 (+ 672us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:14.991337 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:14.991342 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:14.992089 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:14.992091 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:14.992949 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:14.992952 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:14.993330 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:14.993331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:14.993512 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:14.994275 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:14.994284 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:14.995892 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:14.997258 (+ 1366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:14.997264 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:14.997266 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:14.997476 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:14.997702 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:14.997703 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:14.997703 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:14.997805 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:14.998054 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:14.998059 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:14.998174 (+ 115us) client_negotiation.cc:769] Sending connection context
0812 16:37:14.998261 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 16:37:14.998376 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":173,"threads_started":1}
I20260812 16:37:14.999143 6605 heartbeater.cc:347] Connected to a master server at 127.0.253.254:37471
I20260812 16:37:14.999264 6605 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:14.999445 6605 heartbeater.cc:511] Master 127.0.253.254:37471 requested a full tablet report, sending...
I20260812 16:37:14.999943 5806 ts_manager.cc:194] Registered new tserver with Master: 174d1019402c4ba8a63147f8c946b8d6 (127.0.253.195:40895)
I20260812 16:37:15.000377 5806 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.195@KRBTEST.COM'} at 127.0.253.195:49487
I20260812 16:37:15.005864 1015 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 16:37:15.019855 5874 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:15.018789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50662 (local address 127.0.253.254:37471)
0812 16:37:15.018864 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:15.018866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:15.018934 (+ 68us) server_negotiation.cc:407] Connection header received
0812 16:37:15.018986 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:15.018987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:15.018997 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:15.019127 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:15.019751 (+ 624us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50662: BlockingRecv error: recv got EOF from 127.0.0.1:50662 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 16:37:15.020669 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 5922
I20260812 16:37:15.044452 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 6163
I20260812 16:37:15.069506 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 6385
I20260812 16:37:15.079348 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 5745
2026-08-12T16:37:15Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2177 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:37:15 dist-test-slave-w4v5 krb5kdc[6625](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:37:15 dist-test-slave-w4v5 krb5kdc[6625](info): set up 2 sockets
Aug 12 16:37:15 dist-test-slave-w4v5 krb5kdc[6625](info): commencing operation
krb5kdc: starting...
W20260812 16:37:16.389348 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.283s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:37:16 dist-test-slave-w4v5 krb5kdc[6625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:37:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:37:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
I20260812 16:37:16.519368 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:38299
--webserver_interface=127.0.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:34723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:38299
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:16.605049 6651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:16.605245 6651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:16.605266 6651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:16.607118 6651 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:37:16.607180 6651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:16.607195 6651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:16.607208 6651 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:37:16.607220 6651 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:37:16.608762 6651 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:34723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:38299
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:38299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:16.609022 6651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:16.609257 6651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:16.614344 6661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:16.614475 6651 server_base.cc:1034] running on GCE node
W20260812 16:37:16.614349 6667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:16.614380 6665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:16.614848 6651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:16.615196 6651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:16.616345 6651 hybrid_clock.cc:648] HybridClock initialized: now 1786552636616331 us; error 28 us; skew 500 ppm
Aug 12 16:37:16 dist-test-slave-w4v5 krb5kdc[6625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552636, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:16.619030 6651 init.cc:377] Logged in from keytab as kudu/127.0.253.254@KRBTEST.COM (short username kudu)
I20260812 16:37:16.619518 6651 webserver.cc:460] Webserver started at http://127.0.253.254:35311/ using document root <none> and password file <none>
I20260812 16:37:16.619791 6651 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:16.619838 6651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:16.620008 6651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:16.621080 6651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "98e4ef022bd14da89c26a691c64ef88a"
format_stamp: "Formatted at 2026-08-12 16:37:16 on dist-test-slave-w4v5"
server_key: "213fecdf1d4f08eabec69def8342b238"
server_key_iv: "fc884636670a44c4cb967cec9e95bc55"
server_key_version: "encryptionkey@0"
I20260812 16:37:16.621449 6651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "98e4ef022bd14da89c26a691c64ef88a"
format_stamp: "Formatted at 2026-08-12 16:37:16 on dist-test-slave-w4v5"
server_key: "213fecdf1d4f08eabec69def8342b238"
server_key_iv: "fc884636670a44c4cb967cec9e95bc55"
server_key_version: "encryptionkey@0"
I20260812 16:37:16.622908 6651 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260812 16:37:16.623818 6681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:16.624023 6651 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:16.624141 6651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "98e4ef022bd14da89c26a691c64ef88a"
format_stamp: "Formatted at 2026-08-12 16:37:16 on dist-test-slave-w4v5"
server_key: "213fecdf1d4f08eabec69def8342b238"
server_key_iv: "fc884636670a44c4cb967cec9e95bc55"
server_key_version: "encryptionkey@0"
I20260812 16:37:16.624235 6651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:16.644882 6651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:16.647406 6651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:16.647588 6651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:16.651800 6651 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:38299
I20260812 16:37:16.651844 6774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:38299 every 8 connection(s)
I20260812 16:37:16.652242 6651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:37:16.653153 6775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:16.655561 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 6651
I20260812 16:37:16.655658 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:37:16.655748 6775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98e4ef022bd14da89c26a691c64ef88a: Bootstrap starting.
I20260812 16:37:16.655843 1015 external_mini_cluster.cc:1428] Setting key 0b15c6f5376522c094ecb7c5a9689812
I20260812 16:37:16.656560 6775 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98e4ef022bd14da89c26a691c64ef88a: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:16.656893 6775 log.cc:824] T 00000000000000000000000000000000 P 98e4ef022bd14da89c26a691c64ef88a: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:16.657578 6775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98e4ef022bd14da89c26a691c64ef88a: No bootstrap required, opened a new log
I20260812 16:37:16.659600 6775 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98e4ef022bd14da89c26a691c64ef88a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98e4ef022bd14da89c26a691c64ef88a" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 38299 } }
I20260812 16:37:16.659734 6775 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98e4ef022bd14da89c26a691c64ef88a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:16.659757 6775 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98e4ef022bd14da89c26a691c64ef88a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98e4ef022bd14da89c26a691c64ef88a, State: Initialized, Role: FOLLOWER
I20260812 16:37:16.659889 6775 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98e4ef022bd14da89c26a691c64ef88a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98e4ef022bd14da89c26a691c64ef88a" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 38299 } }
I20260812 16:37:16.659976 6775 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98e4ef022bd14da89c26a691c64ef88a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:37:16.660009 6775 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98e4ef022bd14da89c26a691c64ef88a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:37:16.660054 6775 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98e4ef022bd14da89c26a691c64ef88a [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:16.660774 6775 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98e4ef022bd14da89c26a691c64ef88a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98e4ef022bd14da89c26a691c64ef88a" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 38299 } }
I20260812 16:37:16.660887 6775 leader_election.cc:302] T 00000000000000000000000000000000 P 98e4ef022bd14da89c26a691c64ef88a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98e4ef022bd14da89c26a691c64ef88a; no voters:
I20260812 16:37:16.661092 6775 leader_election.cc:288] T 00000000000000000000000000000000 P 98e4ef022bd14da89c26a691c64ef88a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:37:16.661127 6784 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98e4ef022bd14da89c26a691c64ef88a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:37:16.661311 6784 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98e4ef022bd14da89c26a691c64ef88a [term 1 LEADER]: Becoming Leader. State: Replica: 98e4ef022bd14da89c26a691c64ef88a, State: Running, Role: LEADER
I20260812 16:37:16.661518 6775 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98e4ef022bd14da89c26a691c64ef88a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:37:16.661484 6784 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98e4ef022bd14da89c26a691c64ef88a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98e4ef022bd14da89c26a691c64ef88a" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 38299 } }
I20260812 16:37:16.662034 6786 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98e4ef022bd14da89c26a691c64ef88a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98e4ef022bd14da89c26a691c64ef88a. Latest consensus state: current_term: 1 leader_uuid: "98e4ef022bd14da89c26a691c64ef88a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98e4ef022bd14da89c26a691c64ef88a" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 38299 } } }
I20260812 16:37:16.662259 6786 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98e4ef022bd14da89c26a691c64ef88a [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:16.662704 6787 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98e4ef022bd14da89c26a691c64ef88a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98e4ef022bd14da89c26a691c64ef88a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98e4ef022bd14da89c26a691c64ef88a" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 38299 } } }
I20260812 16:37:16.662797 6787 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98e4ef022bd14da89c26a691c64ef88a [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:16.662933 6799 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 12 16:37:16 dist-test-slave-w4v5 krb5kdc[6625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:16.663506 6799 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:37:16.665400 6799 catalog_manager.cc:1347] Generated new cluster ID: 8ab6555de223465689519d9559f5464e
I20260812 16:37:16.665460 6799 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:37:16.667344 6780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:16.656367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40640 (local address 127.0.253.254:38299)
0812 16:37:16.656554 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:16.656561 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:16.656588 (+ 27us) server_negotiation.cc:407] Connection header received
0812 16:37:16.657128 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:16.657136 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:16.657187 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:16.657349 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:16.658079 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:16.659052 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:16.659942 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:16.660572 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:16.660725 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:16.663593 (+ 2868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:16.663597 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:16.663598 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:16.663611 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:16.665653 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:16.666067 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:16.666070 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:16.666070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:16.666188 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:16.666411 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:16.666412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:16.666412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:16.666697 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:16.666829 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:16.666975 (+ 146us) server_negotiation.cc:299] Negotiation successful
0812 16:37:16.667132 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":95,"threads_started":1}
I20260812 16:37:16.668179 1015 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:37:16.680413 6799 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:37:16.681249 6799 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:37:16.687595 6799 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98e4ef022bd14da89c26a691c64ef88a: Generated new TSK 0
I20260812 16:37:16.687767 6799 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:37:16.689718 6780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:16.685457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40648 (local address 127.0.253.254:38299)
0812 16:37:16.685500 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:16.685502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:16.685627 (+ 125us) server_negotiation.cc:407] Connection header received
0812 16:37:16.685667 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:16.685668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:16.685679 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:16.685758 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:16.686169 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:16.686976 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:16.688033 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:16.688378 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:16.688454 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:16.689197 (+ 743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:16.689198 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:16.689199 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:37:16.689429 (+ 230us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:16.689449 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:37:16.689457 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:16.689521 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:16.689610 (+ 89us) server_negotiation.cc:299] Negotiation successful
0812 16:37:16.689639 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 16:37:16.692453 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 6651
2026-08-12T16:37:16Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1616 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:37:16 dist-test-slave-w4v5 krb5kdc[6839](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:37:16 dist-test-slave-w4v5 krb5kdc[6839](info): set up 2 sockets
Aug 12 16:37:16 dist-test-slave-w4v5 krb5kdc[6839](info): commencing operation
krb5kdc: starting...
W20260812 16:37:18.023695 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.300s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:37:18 dist-test-slave-w4v5 krb5kdc[6839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:37:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:37:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
I20260812 16:37:18.158571 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:40081
--webserver_interface=127.0.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:44111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:40081
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:18.243963 6865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:18.244191 6865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:18.244215 6865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:18.245402 6865 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:37:18.245447 6865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:18.245462 6865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:18.245476 6865 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:37:18.245487 6865 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:37:18.246992 6865 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:44111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:40081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:40081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:18.247354 6865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:18.247601 6865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:18.252807 6877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:18.252799 6875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:18.252835 6865 server_base.cc:1034] running on GCE node
W20260812 16:37:18.252797 6874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:18.253269 6865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:18.253582 6865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:18.254779 6865 hybrid_clock.cc:648] HybridClock initialized: now 1786552638254748 us; error 52 us; skew 500 ppm
Aug 12 16:37:18 dist-test-slave-w4v5 krb5kdc[6839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552638, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:18.257562 6865 init.cc:377] Logged in from keytab as kudu/127.0.253.254@KRBTEST.COM (short username kudu)
I20260812 16:37:18.258086 6865 webserver.cc:460] Webserver started at http://127.0.253.254:33063/ using document root <none> and password file <none>
I20260812 16:37:18.258340 6865 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:18.258409 6865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:18.258561 6865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:18.259724 6865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "147df755513b44d1a72453495f7e42be"
format_stamp: "Formatted at 2026-08-12 16:37:18 on dist-test-slave-w4v5"
server_key: "ad0622fe54c3e12d5e102fe674dfadde"
server_key_iv: "4b531bb4474cdd160384aecbf8cefe65"
server_key_version: "encryptionkey@0"
I20260812 16:37:18.260157 6865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "147df755513b44d1a72453495f7e42be"
format_stamp: "Formatted at 2026-08-12 16:37:18 on dist-test-slave-w4v5"
server_key: "ad0622fe54c3e12d5e102fe674dfadde"
server_key_iv: "4b531bb4474cdd160384aecbf8cefe65"
server_key_version: "encryptionkey@0"
I20260812 16:37:18.261569 6865 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:37:18.262434 6887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:18.262653 6865 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:18.262771 6865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "147df755513b44d1a72453495f7e42be"
format_stamp: "Formatted at 2026-08-12 16:37:18 on dist-test-slave-w4v5"
server_key: "ad0622fe54c3e12d5e102fe674dfadde"
server_key_iv: "4b531bb4474cdd160384aecbf8cefe65"
server_key_version: "encryptionkey@0"
I20260812 16:37:18.262850 6865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:18.286697 6865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:18.288856 6865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:18.289038 6865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:18.293377 6865 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:40081
I20260812 16:37:18.293416 6983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:40081 every 8 connection(s)
I20260812 16:37:18.293817 6865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:37:18.294492 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 6865
I20260812 16:37:18.294589 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:37:18.294809 1015 external_mini_cluster.cc:1428] Setting key 872c08d47ee9cb07743a05cc5ef587f4
I20260812 16:37:18.295360 6984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:18.298815 6984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 147df755513b44d1a72453495f7e42be: Bootstrap starting.
I20260812 16:37:18.299562 6984 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 147df755513b44d1a72453495f7e42be: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:18.299886 6984 log.cc:824] T 00000000000000000000000000000000 P 147df755513b44d1a72453495f7e42be: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:18.300803 6984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 147df755513b44d1a72453495f7e42be: No bootstrap required, opened a new log
Aug 12 16:37:18 dist-test-slave-w4v5 krb5kdc[6839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:18.303072 6984 raft_consensus.cc:348] T 00000000000000000000000000000000 P 147df755513b44d1a72453495f7e42be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "147df755513b44d1a72453495f7e42be" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 40081 } }
I20260812 16:37:18.303283 6984 raft_consensus.cc:374] T 00000000000000000000000000000000 P 147df755513b44d1a72453495f7e42be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:18.303332 6984 raft_consensus.cc:729] T 00000000000000000000000000000000 P 147df755513b44d1a72453495f7e42be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147df755513b44d1a72453495f7e42be, State: Initialized, Role: FOLLOWER
I20260812 16:37:18.303498 6984 consensus_queue.cc:260] T 00000000000000000000000000000000 P 147df755513b44d1a72453495f7e42be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "147df755513b44d1a72453495f7e42be" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 40081 } }
I20260812 16:37:18.303572 6984 raft_consensus.cc:388] T 00000000000000000000000000000000 P 147df755513b44d1a72453495f7e42be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:37:18.303596 6984 raft_consensus.cc:482] T 00000000000000000000000000000000 P 147df755513b44d1a72453495f7e42be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:37:18.303661 6984 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 147df755513b44d1a72453495f7e42be [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:18.304383 6984 raft_consensus.cc:504] T 00000000000000000000000000000000 P 147df755513b44d1a72453495f7e42be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "147df755513b44d1a72453495f7e42be" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 40081 } }
I20260812 16:37:18.304477 6984 leader_election.cc:302] T 00000000000000000000000000000000 P 147df755513b44d1a72453495f7e42be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 147df755513b44d1a72453495f7e42be; no voters:
I20260812 16:37:18.304675 6984 leader_election.cc:288] T 00000000000000000000000000000000 P 147df755513b44d1a72453495f7e42be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:37:18.304708 6997 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 147df755513b44d1a72453495f7e42be [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:37:18.304842 6997 raft_consensus.cc:686] T 00000000000000000000000000000000 P 147df755513b44d1a72453495f7e42be [term 1 LEADER]: Becoming Leader. State: Replica: 147df755513b44d1a72453495f7e42be, State: Running, Role: LEADER
I20260812 16:37:18.304993 6984 sys_catalog.cc:564] T 00000000000000000000000000000000 P 147df755513b44d1a72453495f7e42be [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:37:18.305120 6997 consensus_queue.cc:237] T 00000000000000000000000000000000 P 147df755513b44d1a72453495f7e42be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "147df755513b44d1a72453495f7e42be" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 40081 } }
I20260812 16:37:18.305576 6999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 147df755513b44d1a72453495f7e42be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "147df755513b44d1a72453495f7e42be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "147df755513b44d1a72453495f7e42be" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 40081 } } }
I20260812 16:37:18.305830 6999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 147df755513b44d1a72453495f7e42be [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:18.305759 7000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 147df755513b44d1a72453495f7e42be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 147df755513b44d1a72453495f7e42be. Latest consensus state: current_term: 1 leader_uuid: "147df755513b44d1a72453495f7e42be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "147df755513b44d1a72453495f7e42be" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 40081 } } }
I20260812 16:37:18.306113 7000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 147df755513b44d1a72453495f7e42be [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:18.306313 7016 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:37:18.306840 6992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:18.295454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37302 (local address 127.0.253.254:40081)
0812 16:37:18.295652 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:18.295661 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:18.295698 (+ 37us) server_negotiation.cc:407] Connection header received
0812 16:37:18.296146 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:18.296153 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:18.296201 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:18.296318 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:18.297192 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:18.298120 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:18.299348 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:18.299852 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:18.299954 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:18.302747 (+ 2793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:18.302752 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:18.302752 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:18.302772 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:18.304925 (+ 2153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:18.305332 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:18.305336 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:18.305336 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:18.305436 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:18.305697 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:18.305698 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:18.305699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:18.306045 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:18.306183 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:18.306474 (+ 291us) server_negotiation.cc:299] Negotiation successful
0812 16:37:18.306641 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":112,"threads_started":1}
I20260812 16:37:18.306847 7016 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:37:18.308529 7016 catalog_manager.cc:1347] Generated new cluster ID: c266bdab38494d7b8aaea0ad7453df0b
I20260812 16:37:18.308585 7016 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:37:18.314695 7016 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:37:18.315552 7016 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:37:18.327919 7016 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 147df755513b44d1a72453495f7e42be: Generated new TSK 0
I20260812 16:37:18.328280 7016 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab.
Entry for principal kudu/127.0.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab.
I20260812 16:37:18.362447 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.193:0
--local_ip_for_outbound_sockets=127.0.253.193
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:40081
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:44111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:18.465148 7045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:18.465407 7045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:18.465451 7045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:18.466868 7045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:18.466930 7045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:18.466977 7045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.193
I20260812 16:37:18.468566 7045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:44111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:40081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.193
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:18.468863 7045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:18.469131 7045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:18.475312 7057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:18.475375 7065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:18.475301 7063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:18.475740 7045 server_base.cc:1034] running on GCE node
I20260812 16:37:18.475952 7045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:18.476244 7045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:18.477427 7045 hybrid_clock.cc:648] HybridClock initialized: now 1786552638477414 us; error 30 us; skew 500 ppm
Aug 12 16:37:18 dist-test-slave-w4v5 krb5kdc[6839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552638, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:18.480123 7045 init.cc:377] Logged in from keytab as kudu/127.0.253.193@KRBTEST.COM (short username kudu)
I20260812 16:37:18.480615 7045 webserver.cc:460] Webserver started at http://127.0.253.193:42663/ using document root <none> and password file <none>
I20260812 16:37:18.480903 7045 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:18.480952 7045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:18.481127 7045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:18.482223 7045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-0/data/instance:
uuid: "f13a1d9ba5c4425ca13f5a9eeff57649"
format_stamp: "Formatted at 2026-08-12 16:37:18 on dist-test-slave-w4v5"
server_key: "a8897d08f25e070b6313215b4c5e072d"
server_key_iv: "9a5221f22c335e79acb9b439a71b3bca"
server_key_version: "encryptionkey@0"
I20260812 16:37:18.482589 7045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance:
uuid: "f13a1d9ba5c4425ca13f5a9eeff57649"
format_stamp: "Formatted at 2026-08-12 16:37:18 on dist-test-slave-w4v5"
server_key: "a8897d08f25e070b6313215b4c5e072d"
server_key_iv: "9a5221f22c335e79acb9b439a71b3bca"
server_key_version: "encryptionkey@0"
I20260812 16:37:18.484082 7045 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:37:18.484925 7075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:18.485299 7045 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:37:18.485441 7045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-0/wal
uuid: "f13a1d9ba5c4425ca13f5a9eeff57649"
format_stamp: "Formatted at 2026-08-12 16:37:18 on dist-test-slave-w4v5"
server_key: "a8897d08f25e070b6313215b4c5e072d"
server_key_iv: "9a5221f22c335e79acb9b439a71b3bca"
server_key_version: "encryptionkey@0"
I20260812 16:37:18.485527 7045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:18.496199 7045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:18.498164 7045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:18.498337 7045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:18.498582 7045 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:37:18.498915 7045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:18.498976 7045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:18.499022 7045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:18.499046 7045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:18.504956 7045 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.193:39863
I20260812 16:37:18.505093 7268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.193:39863 every 8 connection(s)
I20260812 16:37:18.505386 7045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
I20260812 16:37:18.509407 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 7045
I20260812 16:37:18.509502 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance
I20260812 16:37:18.509707 1015 external_mini_cluster.cc:1428] Setting key 82a35722d8742d2149390b7166742d07
Aug 12 16:37:18 dist-test-slave-w4v5 krb5kdc[6839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552638, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:18.515566 6992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:18.506131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:50317 (local address 127.0.253.254:40081)
0812 16:37:18.506186 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:18.506189 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:18.506422 (+ 233us) server_negotiation.cc:407] Connection header received
0812 16:37:18.506895 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:18.506896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:18.506907 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:18.506961 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:18.507968 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:18.508473 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:18.509611 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:18.509946 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:18.510011 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:18.512609 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:18.512611 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:18.512611 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:18.512619 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:18.514034 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:18.514469 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:18.514470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:18.514471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:18.514536 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:18.514801 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:18.514802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:18.514802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:18.514925 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:18.514999 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:18.515375 (+ 376us) server_negotiation.cc:299] Negotiation successful
0812 16:37:18.515474 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 16:37:18.515771 7274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:18.506100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:40081 (local address 127.0.253.193:50317)
0812 16:37:18.506293 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:18.506332 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:18.506814 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:18.507020 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:18.507024 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:18.507687 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:18.507885 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:18.507891 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:18.508598 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:18.508600 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:18.509445 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:18.509447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:18.510059 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:18.510060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:18.510210 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:18.510777 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:18.510788 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:18.512506 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:18.514139 (+ 1633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:18.514147 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:18.514150 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:18.514388 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:18.514606 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:18.514607 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:18.514607 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:18.514731 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:18.515001 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:18.515009 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:18.515230 (+ 221us) client_negotiation.cc:769] Sending connection context
0812 16:37:18.515339 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 16:37:18.515507 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":94,"threads_started":1}
I20260812 16:37:18.516348 7270 heartbeater.cc:347] Connected to a master server at 127.0.253.254:40081
I20260812 16:37:18.516469 7270 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:18.516692 7270 heartbeater.cc:511] Master 127.0.253.254:40081 requested a full tablet report, sending...
I20260812 16:37:18.517273 6914 ts_manager.cc:194] Registered new tserver with Master: f13a1d9ba5c4425ca13f5a9eeff57649 (127.0.253.193:39863)
I20260812 16:37:18.517933 6914 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.193@KRBTEST.COM'} at 127.0.253.193:50317
WARNING: no policy specified for kudu/127.0.253.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab.
Entry for principal kudu/127.0.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab.
I20260812 16:37:18.538561 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.194:0
--local_ip_for_outbound_sockets=127.0.253.194
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:40081
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:44111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:18.623332 7277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:18.623605 7277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:18.623634 7277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:18.624740 7277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:18.624776 7277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:18.624802 7277 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.194
I20260812 16:37:18.626186 7277 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:44111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:40081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.194
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:18.626369 7277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:18.626546 7277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:18.631556 7286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:18.631568 7292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:18.631659 7285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:18.632210 7277 server_base.cc:1034] running on GCE node
I20260812 16:37:18.632421 7277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:18.632637 7277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:18.633837 7277 hybrid_clock.cc:648] HybridClock initialized: now 1786552638633814 us; error 39 us; skew 500 ppm
Aug 12 16:37:18 dist-test-slave-w4v5 krb5kdc[6839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552638, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:18.636610 7277 init.cc:377] Logged in from keytab as kudu/127.0.253.194@KRBTEST.COM (short username kudu)
I20260812 16:37:18.637115 7277 webserver.cc:460] Webserver started at http://127.0.253.194:34055/ using document root <none> and password file <none>
I20260812 16:37:18.637368 7277 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:18.637447 7277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:18.637625 7277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:18.638795 7277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-1/data/instance:
uuid: "88418348029d493189fc2f41f2fab7a2"
format_stamp: "Formatted at 2026-08-12 16:37:18 on dist-test-slave-w4v5"
server_key: "4c92bc4081ccfd8b6b3dae55ead5dfee"
server_key_iv: "6641ef6e17a1bfc1fa0ca21a5c3828e6"
server_key_version: "encryptionkey@0"
I20260812 16:37:18.639248 7277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance:
uuid: "88418348029d493189fc2f41f2fab7a2"
format_stamp: "Formatted at 2026-08-12 16:37:18 on dist-test-slave-w4v5"
server_key: "4c92bc4081ccfd8b6b3dae55ead5dfee"
server_key_iv: "6641ef6e17a1bfc1fa0ca21a5c3828e6"
server_key_version: "encryptionkey@0"
I20260812 16:37:18.640754 7277 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:37:18.641616 7301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:18.641884 7277 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:18.642004 7277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-1/wal
uuid: "88418348029d493189fc2f41f2fab7a2"
format_stamp: "Formatted at 2026-08-12 16:37:18 on dist-test-slave-w4v5"
server_key: "4c92bc4081ccfd8b6b3dae55ead5dfee"
server_key_iv: "6641ef6e17a1bfc1fa0ca21a5c3828e6"
server_key_version: "encryptionkey@0"
I20260812 16:37:18.642086 7277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:18.650651 7277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:18.653025 7277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:18.653190 7277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:18.653429 7277 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:37:18.653754 7277 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:18.653810 7277 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:18.653846 7277 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:18.653873 7277 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:18.660429 7277 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.194:43225
I20260812 16:37:18.660478 7489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.194:43225 every 8 connection(s)
I20260812 16:37:18.660813 7277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
I20260812 16:37:18.664317 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 7277
I20260812 16:37:18.664441 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance
I20260812 16:37:18.664654 1015 external_mini_cluster.cc:1428] Setting key 66b8966aabe6d7a14117847fc0fff5c4
Aug 12 16:37:18 dist-test-slave-w4v5 krb5kdc[6839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552638, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:18.674512 6992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:18.661591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:53615 (local address 127.0.253.254:40081)
0812 16:37:18.661682 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:18.661684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:18.661879 (+ 195us) server_negotiation.cc:407] Connection header received
0812 16:37:18.662357 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:18.662358 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:18.662369 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:18.662431 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:18.663443 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:18.664093 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:18.665429 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:18.665703 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:18.665756 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:18.669659 (+ 3903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:18.669660 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:18.669660 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:18.669668 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:18.672555 (+ 2887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:18.673217 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:18.673219 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:18.673219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:18.673294 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:18.673614 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:18.673615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:18.673616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:18.673777 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:18.673848 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:18.674294 (+ 446us) server_negotiation.cc:299] Negotiation successful
0812 16:37:18.674386 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 16:37:18.674664 7493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:18.661556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:40081 (local address 127.0.253.194:53615)
0812 16:37:18.661753 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:18.661790 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:18.662266 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:18.662491 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:18.662494 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:18.663145 (+ 651us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:18.663354 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:18.663359 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:18.664225 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:18.664226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:18.665352 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:18.665355 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:18.665770 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:18.665772 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:18.665914 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:18.666624 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:18.666636 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:18.669529 (+ 2893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:18.672753 (+ 3224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:18.672763 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:18.672766 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:18.673113 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:18.673358 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:18.673359 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:18.673360 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:18.673527 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:18.673884 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:18.673892 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:18.674089 (+ 197us) client_negotiation.cc:769] Sending connection context
0812 16:37:18.674216 (+ 127us) client_negotiation.cc:240] Negotiation successful
0812 16:37:18.674414 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":130,"threads_started":1}
I20260812 16:37:18.675422 7491 heartbeater.cc:347] Connected to a master server at 127.0.253.254:40081
I20260812 16:37:18.675539 7491 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.0.253.195@KRBTEST.COM; defaulting to no policy
I20260812 16:37:18.675760 7491 heartbeater.cc:511] Master 127.0.253.254:40081 requested a full tablet report, sending...
I20260812 16:37:18.676239 6914 ts_manager.cc:194] Registered new tserver with Master: 88418348029d493189fc2f41f2fab7a2 (127.0.253.194:43225)
I20260812 16:37:18.676716 6914 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.194@KRBTEST.COM'} at 127.0.253.194:53615
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab.
Entry for principal kudu/127.0.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab.
I20260812 16:37:18.700273 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.195:0
--local_ip_for_outbound_sockets=127.0.253.195
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:40081
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:44111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:18.803433 7498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:18.803639 7498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:18.803664 7498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:18.804936 7498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:18.804979 7498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:18.805006 7498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.195
I20260812 16:37:18.806793 7498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:44111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:40081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.195
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:18.807044 7498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:18.807363 7498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:18.813297 7509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:18.813300 7513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:18.813300 7515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:18.813628 7498 server_base.cc:1034] running on GCE node
I20260812 16:37:18.813884 7498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:18.814153 7498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:18.815315 7498 hybrid_clock.cc:648] HybridClock initialized: now 1786552638815299 us; error 32 us; skew 500 ppm
Aug 12 16:37:18 dist-test-slave-w4v5 krb5kdc[6839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552638, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:18.818380 7498 init.cc:377] Logged in from keytab as kudu/127.0.253.195@KRBTEST.COM (short username kudu)
I20260812 16:37:18.819003 7498 webserver.cc:460] Webserver started at http://127.0.253.195:37341/ using document root <none> and password file <none>
I20260812 16:37:18.819289 7498 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:18.819357 7498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:18.819499 7498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:18.820988 7498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-2/data/instance:
uuid: "44b1ee789b9846a0813faff43ae637e1"
format_stamp: "Formatted at 2026-08-12 16:37:18 on dist-test-slave-w4v5"
server_key: "22c6e37f1e09b6033a181a809d837095"
server_key_iv: "ba17553bf2083779ff44440e79b7054b"
server_key_version: "encryptionkey@0"
I20260812 16:37:18.821506 7498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance:
uuid: "44b1ee789b9846a0813faff43ae637e1"
format_stamp: "Formatted at 2026-08-12 16:37:18 on dist-test-slave-w4v5"
server_key: "22c6e37f1e09b6033a181a809d837095"
server_key_iv: "ba17553bf2083779ff44440e79b7054b"
server_key_version: "encryptionkey@0"
I20260812 16:37:18.823026 7498 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260812 16:37:18.823968 7523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:18.824234 7498 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:18.824352 7498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-2/wal
uuid: "44b1ee789b9846a0813faff43ae637e1"
format_stamp: "Formatted at 2026-08-12 16:37:18 on dist-test-slave-w4v5"
server_key: "22c6e37f1e09b6033a181a809d837095"
server_key_iv: "ba17553bf2083779ff44440e79b7054b"
server_key_version: "encryptionkey@0"
I20260812 16:37:18.824445 7498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:18.870595 7498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:18.872896 7498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:18.873068 7498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:18.873354 7498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:37:18.873689 7498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:18.873723 7498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:18.873754 7498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:18.873802 7498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:18.880029 7498 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.195:34943
I20260812 16:37:18.880076 7717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.195:34943 every 8 connection(s)
I20260812 16:37:18.880424 7498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
Aug 12 16:37:18 dist-test-slave-w4v5 krb5kdc[6839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552638, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:18.888283 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 7498
I20260812 16:37:18.888401 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance
I20260812 16:37:18.888602 1015 external_mini_cluster.cc:1428] Setting key 08ecc95534239c29103230aab7a95abf
I20260812 16:37:18.890250 6992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:18.881290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:41681 (local address 127.0.253.254:40081)
0812 16:37:18.881338 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:18.881342 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:18.881530 (+ 188us) server_negotiation.cc:407] Connection header received
0812 16:37:18.882090 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:18.882091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:18.882101 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:18.882178 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:18.883206 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:18.883696 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:18.884683 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:18.884862 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:18.884914 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:18.887152 (+ 2238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:18.887153 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:18.887154 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:18.887160 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:18.888687 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:18.889215 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:18.889216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:18.889216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:18.889291 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:18.889627 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:18.889628 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:18.889628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:18.889756 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:18.889814 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:18.890110 (+ 296us) server_negotiation.cc:299] Negotiation successful
0812 16:37:18.890186 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 16:37:18.890364 7723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:18.881221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:40081 (local address 127.0.253.195:41681)
0812 16:37:18.881407 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:18.881452 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:18.882001 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:18.882238 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:18.882241 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:18.882848 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:18.883046 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:18.883051 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:18.883779 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:18.883780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:18.884606 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:18.884608 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:18.884936 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:18.884937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:18.885037 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:18.885531 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:18.885540 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:18.887001 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:18.888759 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:18.888767 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:18.888769 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:18.889098 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:18.889388 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:18.889389 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:18.889389 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:18.889537 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:18.889818 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:18.889823 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:18.889943 (+ 120us) client_negotiation.cc:769] Sending connection context
0812 16:37:18.890034 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 16:37:18.890173 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":105,"threads_started":1}
I20260812 16:37:18.891058 7720 heartbeater.cc:347] Connected to a master server at 127.0.253.254:40081
I20260812 16:37:18.891203 7720 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:18.891361 7720 heartbeater.cc:511] Master 127.0.253.254:40081 requested a full tablet report, sending...
I20260812 16:37:18.891681 6914 ts_manager.cc:194] Registered new tserver with Master: 44b1ee789b9846a0813faff43ae637e1 (127.0.253.195:34943)
I20260812 16:37:18.892069 6914 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.195@KRBTEST.COM'} at 127.0.253.195:41681
I20260812 16:37:18.900784 1015 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 16:37:18 dist-test-slave-w4v5 krb5kdc[6839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552638, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 12 16:37:18 dist-test-slave-w4v5 krb5kdc[6839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552638, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:18.925644 6992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:18.918575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37306 (local address 127.0.253.254:40081)
0812 16:37:18.918657 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:18.918659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:18.918796 (+ 137us) server_negotiation.cc:407] Connection header received
0812 16:37:18.918830 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:18.918831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:18.918843 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:18.918891 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:18.919537 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:18.920035 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:18.920780 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:18.920982 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:18.921041 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:18.923446 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:18.923448 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:18.923448 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:18.923455 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:18.924683 (+ 1228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:18.924984 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:18.924985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:18.924985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:18.925037 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:18.925212 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:18.925212 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:18.925213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:18.925358 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:18.925436 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:18.925516 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 16:37:18.925576 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260812 16:37:18.925886 6914 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:37306
I20260812 16:37:18.926313 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 7045
I20260812 16:37:18.938448 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 7277
I20260812 16:37:18.953336 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 7498
I20260812 16:37:18.981065 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 6865
2026-08-12T16:37:18Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2286 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:37:19 dist-test-slave-w4v5 krb5kdc[7749](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:37:19 dist-test-slave-w4v5 krb5kdc[7749](info): set up 2 sockets
Aug 12 16:37:19 dist-test-slave-w4v5 krb5kdc[7749](info): commencing operation
krb5kdc: starting...
W20260812 16:37:20.294317 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.284s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:37:20 dist-test-slave-w4v5 krb5kdc[7749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552640, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:37:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:37:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
I20260812 16:37:20.422243 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:35777
--webserver_interface=127.0.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:33151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:35777
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:20.506996 7771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:20.507283 7771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:20.507308 7771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:20.508540 7771 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:37:20.508590 7771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:20.508606 7771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:20.508620 7771 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:37:20.508631 7771 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:37:20.510681 7771 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:33151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:35777
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:35777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:20.511023 7771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:20.511319 7771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:20.516561 7779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:20.516570 7781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:20.516624 7771 server_base.cc:1034] running on GCE node
W20260812 16:37:20.516590 7778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:20.517122 7771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:20.517467 7771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:20.518642 7771 hybrid_clock.cc:648] HybridClock initialized: now 1786552640518633 us; error 36 us; skew 500 ppm
Aug 12 16:37:20 dist-test-slave-w4v5 krb5kdc[7749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552640, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:20.525830 7771 init.cc:377] Logged in from keytab as kudu/127.0.253.254@KRBTEST.COM (short username kudu)
I20260812 16:37:20.526367 7771 webserver.cc:460] Webserver started at http://127.0.253.254:35325/ using document root <none> and password file <none>
I20260812 16:37:20.526620 7771 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:20.526695 7771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:20.526851 7771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:20.528072 7771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "3160ad999ab344adab3f5901db9593d1"
format_stamp: "Formatted at 2026-08-12 16:37:20 on dist-test-slave-w4v5"
server_key: "b7a682d58bc17546d9c1c4c41ad813d5"
server_key_iv: "b451155af4b2248ae8c1d2eb15c7bb60"
server_key_version: "encryptionkey@0"
I20260812 16:37:20.528458 7771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "3160ad999ab344adab3f5901db9593d1"
format_stamp: "Formatted at 2026-08-12 16:37:20 on dist-test-slave-w4v5"
server_key: "b7a682d58bc17546d9c1c4c41ad813d5"
server_key_iv: "b451155af4b2248ae8c1d2eb15c7bb60"
server_key_version: "encryptionkey@0"
I20260812 16:37:20.529937 7771 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 16:37:20.530886 7789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:20.531237 7771 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:20.531358 7771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "3160ad999ab344adab3f5901db9593d1"
format_stamp: "Formatted at 2026-08-12 16:37:20 on dist-test-slave-w4v5"
server_key: "b7a682d58bc17546d9c1c4c41ad813d5"
server_key_iv: "b451155af4b2248ae8c1d2eb15c7bb60"
server_key_version: "encryptionkey@0"
I20260812 16:37:20.531451 7771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:20.543342 7771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:20.545266 7771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:20.545447 7771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:20.550156 7771 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:35777
I20260812 16:37:20.550222 7871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:35777 every 8 connection(s)
I20260812 16:37:20.550899 7771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:37:20.551702 7872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:20.553953 7872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3160ad999ab344adab3f5901db9593d1: Bootstrap starting.
I20260812 16:37:20.554544 7872 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3160ad999ab344adab3f5901db9593d1: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:20.554793 7872 log.cc:824] T 00000000000000000000000000000000 P 3160ad999ab344adab3f5901db9593d1: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:20.555600 7872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3160ad999ab344adab3f5901db9593d1: No bootstrap required, opened a new log
I20260812 16:37:20.557163 7872 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3160ad999ab344adab3f5901db9593d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3160ad999ab344adab3f5901db9593d1" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 35777 } }
I20260812 16:37:20.557307 7872 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3160ad999ab344adab3f5901db9593d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:20.557329 7872 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3160ad999ab344adab3f5901db9593d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3160ad999ab344adab3f5901db9593d1, State: Initialized, Role: FOLLOWER
I20260812 16:37:20.557438 7872 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3160ad999ab344adab3f5901db9593d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3160ad999ab344adab3f5901db9593d1" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 35777 } }
I20260812 16:37:20.557574 7872 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3160ad999ab344adab3f5901db9593d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:37:20.557618 7872 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3160ad999ab344adab3f5901db9593d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:37:20.557670 7872 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3160ad999ab344adab3f5901db9593d1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:20.558014 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 7771
I20260812 16:37:20.558149 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:37:20.558348 1015 external_mini_cluster.cc:1428] Setting key 9d8ca8ffa1eb5f6cf3ebeeee30f239ff
I20260812 16:37:20.558947 7872 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3160ad999ab344adab3f5901db9593d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3160ad999ab344adab3f5901db9593d1" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 35777 } }
I20260812 16:37:20.559084 7872 leader_election.cc:302] T 00000000000000000000000000000000 P 3160ad999ab344adab3f5901db9593d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3160ad999ab344adab3f5901db9593d1; no voters:
I20260812 16:37:20.559290 7872 leader_election.cc:288] T 00000000000000000000000000000000 P 3160ad999ab344adab3f5901db9593d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:37:20.559322 7881 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3160ad999ab344adab3f5901db9593d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:37:20.559484 7881 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3160ad999ab344adab3f5901db9593d1 [term 1 LEADER]: Becoming Leader. State: Replica: 3160ad999ab344adab3f5901db9593d1, State: Running, Role: LEADER
I20260812 16:37:20.559581 7872 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3160ad999ab344adab3f5901db9593d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:37:20.559648 7881 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3160ad999ab344adab3f5901db9593d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3160ad999ab344adab3f5901db9593d1" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 35777 } }
I20260812 16:37:20.560021 7881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3160ad999ab344adab3f5901db9593d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3160ad999ab344adab3f5901db9593d1. Latest consensus state: current_term: 1 leader_uuid: "3160ad999ab344adab3f5901db9593d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3160ad999ab344adab3f5901db9593d1" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 35777 } } }
I20260812 16:37:20.560091 7882 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3160ad999ab344adab3f5901db9593d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3160ad999ab344adab3f5901db9593d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3160ad999ab344adab3f5901db9593d1" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 35777 } } }
I20260812 16:37:20.560133 7881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3160ad999ab344adab3f5901db9593d1 [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:20.560148 7882 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3160ad999ab344adab3f5901db9593d1 [sys.catalog]: This master's current role is: LEADER
W20260812 16:37:20.561689 7903 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3160ad999ab344adab3f5901db9593d1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 16:37:20.561754 7903 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:37:20.562414 7888 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:37:20.562636 7888 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:37:20.564445 7888 catalog_manager.cc:1347] Generated new cluster ID: 8122be9637d64df9acea7e57594cd9e9
I20260812 16:37:20.564522 7888 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 16:37:20 dist-test-slave-w4v5 krb5kdc[7749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552640, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:20.571501 7880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:20.558952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49768 (local address 127.0.253.254:35777)
0812 16:37:20.559544 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:20.559556 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:20.559597 (+ 41us) server_negotiation.cc:407] Connection header received
0812 16:37:20.559725 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:20.559734 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:20.559798 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:20.559957 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:20.560843 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:20.562284 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:20.563378 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:20.564059 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:20.564190 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:20.567213 (+ 3023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:20.567218 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:20.567218 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:20.567236 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:20.569448 (+ 2212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:20.570013 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:20.570017 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:20.570017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:20.570170 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:20.570408 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:20.570409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:20.570409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:20.570766 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:20.570922 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:20.571064 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 16:37:20.571251 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":151,"threads_started":1}
I20260812 16:37:20.588361 7888 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:37:20.588984 7888 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:37:20.595306 7888 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3160ad999ab344adab3f5901db9593d1: Generated new TSK 0
I20260812 16:37:20.595521 7888 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab.
Entry for principal kudu/127.0.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab.
I20260812 16:37:20.629861 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.193:0
--local_ip_for_outbound_sockets=127.0.253.193
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:35777
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:33151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:20.714828 7911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:20.715040 7911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:20.715065 7911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:20.716253 7911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:20.716296 7911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:20.716322 7911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.193
I20260812 16:37:20.717854 7911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:33151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:35777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.193
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:20.718103 7911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:20.718346 7911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:20.723562 7927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:20.723578 7921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:20.723562 7920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:20.723610 7911 server_base.cc:1034] running on GCE node
I20260812 16:37:20.724020 7911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:20.724261 7911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:20.725438 7911 hybrid_clock.cc:648] HybridClock initialized: now 1786552640725420 us; error 46 us; skew 500 ppm
Aug 12 16:37:20 dist-test-slave-w4v5 krb5kdc[7749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552640, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:20.728153 7911 init.cc:377] Logged in from keytab as kudu/127.0.253.193@KRBTEST.COM (short username kudu)
I20260812 16:37:20.728559 7911 webserver.cc:460] Webserver started at http://127.0.253.193:41141/ using document root <none> and password file <none>
I20260812 16:37:20.728746 7911 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:20.728787 7911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:20.728878 7911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:20.729885 7911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-0/data/instance:
uuid: "a7b68c87e5274ca79533e657eb50d2c3"
format_stamp: "Formatted at 2026-08-12 16:37:20 on dist-test-slave-w4v5"
server_key: "9bc5fb9d28b051799e90ff86e7ee0f37"
server_key_iv: "a1c79a5a21a049e241a16b78e65dc336"
server_key_version: "encryptionkey@0"
I20260812 16:37:20.730202 7911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance:
uuid: "a7b68c87e5274ca79533e657eb50d2c3"
format_stamp: "Formatted at 2026-08-12 16:37:20 on dist-test-slave-w4v5"
server_key: "9bc5fb9d28b051799e90ff86e7ee0f37"
server_key_iv: "a1c79a5a21a049e241a16b78e65dc336"
server_key_version: "encryptionkey@0"
I20260812 16:37:20.731583 7911 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:37:20.732342 7941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:20.732594 7911 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:37:20.732674 7911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-0/wal
uuid: "a7b68c87e5274ca79533e657eb50d2c3"
format_stamp: "Formatted at 2026-08-12 16:37:20 on dist-test-slave-w4v5"
server_key: "9bc5fb9d28b051799e90ff86e7ee0f37"
server_key_iv: "a1c79a5a21a049e241a16b78e65dc336"
server_key_version: "encryptionkey@0"
I20260812 16:37:20.732729 7911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:20.744956 7911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:20.747254 7911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:20.747377 7911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:20.747567 7911 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:37:20.747924 7911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:20.747961 7911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:20.747984 7911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:20.748034 7911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:20.754065 7911 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.193:44123
I20260812 16:37:20.754206 8146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.193:44123 every 8 connection(s)
I20260812 16:37:20.754508 7911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
I20260812 16:37:20.755945 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 7911
I20260812 16:37:20.756068 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance
I20260812 16:37:20.756258 1015 external_mini_cluster.cc:1428] Setting key b1efd1b7029a7b53b4bad5accdc4251d
Aug 12 16:37:20 dist-test-slave-w4v5 krb5kdc[7749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552640, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.253.194@KRBTEST.COM; defaulting to no policy
I20260812 16:37:20.768419 7880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:20.755351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:52545 (local address 127.0.253.254:35777)
0812 16:37:20.755438 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:20.755440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:20.755654 (+ 214us) server_negotiation.cc:407] Connection header received
0812 16:37:20.756998 (+ 1344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:20.756999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:20.757016 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:20.757116 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:20.758120 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:20.759222 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:20.761065 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:20.761417 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:20.761485 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:20.764450 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:20.764452 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:20.764453 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:20.764465 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:20.766867 (+ 2402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:20.767392 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:20.767394 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:20.767395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:20.767461 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:20.767691 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:20.767692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:20.767692 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:20.767846 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:20.767928 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:20.768252 (+ 324us) server_negotiation.cc:299] Negotiation successful
0812 16:37:20.768351 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 16:37:20.768558 8151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:20.755308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:35777 (local address 127.0.253.193:52545)
0812 16:37:20.755508 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:20.755555 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:20.756818 (+ 1263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:20.757278 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:20.757283 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:20.757693 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:20.758000 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:20.758007 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:20.759371 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:20.759373 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:20.760957 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:20.760961 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:20.761563 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:20.761564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:20.761716 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:20.762341 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:20.762353 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:20.764299 (+ 1946us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:20.766963 (+ 2664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:20.766972 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:20.766975 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:20.767304 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:20.767533 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:20.767534 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:20.767534 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:20.767628 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:20.767941 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:20.767950 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:20.768105 (+ 155us) client_negotiation.cc:769] Sending connection context
0812 16:37:20.768198 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 16:37:20.768332 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":99,"threads_started":1}
I20260812 16:37:20.769163 8148 heartbeater.cc:347] Connected to a master server at 127.0.253.254:35777
I20260812 16:37:20.769282 8148 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:20.769488 8148 heartbeater.cc:511] Master 127.0.253.254:35777 requested a full tablet report, sending...
I20260812 16:37:20.770217 7817 ts_manager.cc:194] Registered new tserver with Master: a7b68c87e5274ca79533e657eb50d2c3 (127.0.253.193:44123)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.194@KRBTEST.COM" created.
I20260812 16:37:20.771023 7817 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.193@KRBTEST.COM'} at 127.0.253.193:52545
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab.
Entry for principal kudu/127.0.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab.
I20260812 16:37:20.786201 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.194:0
--local_ip_for_outbound_sockets=127.0.253.194
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:35777
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:33151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:20.872777 8156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:20.872991 8156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:20.873015 8156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:20.874178 8156 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:20.874223 8156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:20.874250 8156 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.194
I20260812 16:37:20.875851 8156 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:33151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:35777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8156
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.194
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:20.876104 8156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:20.876363 8156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:20.881698 8166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:20.881824 8156 server_base.cc:1034] running on GCE node
W20260812 16:37:20.881768 8169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:20.881700 8173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:20.882100 8156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:20.882380 8156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:20.883543 8156 hybrid_clock.cc:648] HybridClock initialized: now 1786552640883531 us; error 48 us; skew 500 ppm
Aug 12 16:37:20 dist-test-slave-w4v5 krb5kdc[7749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552640, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:20.886197 8156 init.cc:377] Logged in from keytab as kudu/127.0.253.194@KRBTEST.COM (short username kudu)
I20260812 16:37:20.886689 8156 webserver.cc:460] Webserver started at http://127.0.253.194:42577/ using document root <none> and password file <none>
I20260812 16:37:20.886924 8156 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:20.886993 8156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:20.887177 8156 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:20.888298 8156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-1/data/instance:
uuid: "9ff6da09d93f49848686e1bfad426c91"
format_stamp: "Formatted at 2026-08-12 16:37:20 on dist-test-slave-w4v5"
server_key: "a03deec8e5d683721a0c985b292d7ccb"
server_key_iv: "157155c65caad080c6674f471b055437"
server_key_version: "encryptionkey@0"
I20260812 16:37:20.888664 8156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance:
uuid: "9ff6da09d93f49848686e1bfad426c91"
format_stamp: "Formatted at 2026-08-12 16:37:20 on dist-test-slave-w4v5"
server_key: "a03deec8e5d683721a0c985b292d7ccb"
server_key_iv: "157155c65caad080c6674f471b055437"
server_key_version: "encryptionkey@0"
I20260812 16:37:20.890081 8156 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:37:20.890950 8182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:20.891177 8156 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:20.891292 8156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-1/wal
uuid: "9ff6da09d93f49848686e1bfad426c91"
format_stamp: "Formatted at 2026-08-12 16:37:20 on dist-test-slave-w4v5"
server_key: "a03deec8e5d683721a0c985b292d7ccb"
server_key_iv: "157155c65caad080c6674f471b055437"
server_key_version: "encryptionkey@0"
I20260812 16:37:20.891371 8156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:20.910544 8156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:20.912811 8156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:20.913002 8156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:20.913269 8156 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:37:20.913626 8156 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:20.913688 8156 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:20.913731 8156 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:20.913760 8156 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:20.920159 8156 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.194:46139
I20260812 16:37:20.920209 8394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.194:46139 every 8 connection(s)
I20260812 16:37:20.920588 8156 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
I20260812 16:37:20.922387 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 8156
I20260812 16:37:20.922482 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance
I20260812 16:37:20.922688 1015 external_mini_cluster.cc:1428] Setting key 8a17c4e2cffca9583026b271030756e1
Aug 12 16:37:20 dist-test-slave-w4v5 krb5kdc[7749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552640, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:20.931154 7880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:20.921439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:51713 (local address 127.0.253.254:35777)
0812 16:37:20.921610 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:20.921612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:20.921833 (+ 221us) server_negotiation.cc:407] Connection header received
0812 16:37:20.922938 (+ 1105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:20.922940 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:20.922954 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:20.923012 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:20.923715 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:20.924232 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:20.925265 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:20.925485 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:20.925574 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:20.927919 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:20.927921 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:20.927922 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:20.927931 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:20.929663 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:20.930079 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:20.930080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:20.930080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:20.930151 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:20.930401 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:20.930402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:20.930402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:20.930581 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:20.930663 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:20.930922 (+ 259us) server_negotiation.cc:299] Negotiation successful
0812 16:37:20.931021 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139}
I20260812 16:37:20.931277 8400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:20.921411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:35777 (local address 127.0.253.194:51713)
0812 16:37:20.921682 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:20.921730 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:20.922825 (+ 1095us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:20.923115 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:20.923120 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:20.923406 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:20.923605 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:20.923610 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:20.924332 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:20.924333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:20.925170 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:20.925172 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:20.925584 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:20.925585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:20.925733 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:20.926314 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:20.926323 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:20.927819 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:20.929755 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:20.929762 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:20.929763 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:20.929997 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:20.930230 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:20.930230 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:20.930231 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:20.930333 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:20.930657 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:20.930663 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:20.930783 (+ 120us) client_negotiation.cc:769] Sending connection context
0812 16:37:20.930869 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 16:37:20.930995 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":97,"threads_started":1}
I20260812 16:37:20.931864 8395 heartbeater.cc:347] Connected to a master server at 127.0.253.254:35777
I20260812 16:37:20.931982 8395 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:20.932176 8395 heartbeater.cc:511] Master 127.0.253.254:35777 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.0.253.195@KRBTEST.COM; defaulting to no policy
I20260812 16:37:20.932607 7817 ts_manager.cc:194] Registered new tserver with Master: 9ff6da09d93f49848686e1bfad426c91 (127.0.253.194:46139)
I20260812 16:37:20.933061 7817 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.194@KRBTEST.COM'} at 127.0.253.194:51713
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab.
Entry for principal kudu/127.0.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab.
I20260812 16:37:20.952314 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.195:0
--local_ip_for_outbound_sockets=127.0.253.195
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:35777
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:33151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:21.037281 8410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:21.037501 8410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:21.037526 8410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:21.038695 8410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:21.038743 8410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:21.038769 8410 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.195
I20260812 16:37:21.040321 8410 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:33151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:35777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.195
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:21.040593 8410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:21.040843 8410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:21.046182 8420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:21.046218 8421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:21.046203 8425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:21.046439 8410 server_base.cc:1034] running on GCE node
I20260812 16:37:21.046737 8410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:21.046954 8410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:21.048106 8410 hybrid_clock.cc:648] HybridClock initialized: now 1786552641048083 us; error 40 us; skew 500 ppm
Aug 12 16:37:21 dist-test-slave-w4v5 krb5kdc[7749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552641, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:21.050856 8410 init.cc:377] Logged in from keytab as kudu/127.0.253.195@KRBTEST.COM (short username kudu)
I20260812 16:37:21.051374 8410 webserver.cc:460] Webserver started at http://127.0.253.195:36429/ using document root <none> and password file <none>
I20260812 16:37:21.051596 8410 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:21.051638 8410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:21.051738 8410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:21.052767 8410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-2/data/instance:
uuid: "f9ca086b5d43407982fc8dcb8987acdd"
format_stamp: "Formatted at 2026-08-12 16:37:21 on dist-test-slave-w4v5"
server_key: "3296d21580c1e571e3f1e6cb793200f5"
server_key_iv: "ae7cb85331caa65cf74f2f0cf477b721"
server_key_version: "encryptionkey@0"
I20260812 16:37:21.053089 8410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance:
uuid: "f9ca086b5d43407982fc8dcb8987acdd"
format_stamp: "Formatted at 2026-08-12 16:37:21 on dist-test-slave-w4v5"
server_key: "3296d21580c1e571e3f1e6cb793200f5"
server_key_iv: "ae7cb85331caa65cf74f2f0cf477b721"
server_key_version: "encryptionkey@0"
I20260812 16:37:21.054422 8410 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:37:21.055217 8435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:21.055393 8410 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:21.055493 8410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-2/wal
uuid: "f9ca086b5d43407982fc8dcb8987acdd"
format_stamp: "Formatted at 2026-08-12 16:37:21 on dist-test-slave-w4v5"
server_key: "3296d21580c1e571e3f1e6cb793200f5"
server_key_iv: "ae7cb85331caa65cf74f2f0cf477b721"
server_key_version: "encryptionkey@0"
I20260812 16:37:21.055552 8410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:21.072597 8410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:21.074966 8410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:21.075166 8410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:21.075416 8410 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:37:21.075773 8410 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:21.075809 8410 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:21.075856 8410 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:21.075884 8410 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:21.081881 8410 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.195:39263
I20260812 16:37:21.081948 8628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.195:39263 every 8 connection(s)
I20260812 16:37:21.082312 8410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
I20260812 16:37:21.088306 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 8410
I20260812 16:37:21.088389 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance
I20260812 16:37:21.088642 1015 external_mini_cluster.cc:1428] Setting key 18bcf83faaebcf5bc9dbcce153182adf
Aug 12 16:37:21 dist-test-slave-w4v5 krb5kdc[7749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552641, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:21.092903 7880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:21.083210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:40543 (local address 127.0.253.254:35777)
0812 16:37:21.083286 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:21.083289 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:21.083520 (+ 231us) server_negotiation.cc:407] Connection header received
0812 16:37:21.084534 (+ 1014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:21.084536 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:21.084547 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:21.084608 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:21.085246 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:21.085737 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:21.086830 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:21.087029 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:21.087213 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:21.089934 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:21.089935 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:21.089935 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:21.089942 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:21.091433 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:21.091849 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:21.091850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:21.091850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:21.091911 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:21.092222 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:21.092224 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:21.092224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:21.092356 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:21.092417 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:21.092757 (+ 340us) server_negotiation.cc:299] Negotiation successful
0812 16:37:21.092839 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 16:37:21.092978 8632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:21.083173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:35777 (local address 127.0.253.195:40543)
0812 16:37:21.083387 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:21.083426 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:21.084410 (+ 984us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:21.084691 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:21.084695 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:21.084961 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:21.085165 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:21.085169 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:21.085837 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:21.085838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:21.086665 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:21.086667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:21.087224 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:21.087225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:21.087345 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:21.087979 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:21.087987 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:21.089854 (+ 1867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:21.091519 (+ 1665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:21.091526 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:21.091528 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:21.091758 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:21.092005 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:21.092006 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:21.092007 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:21.092128 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:21.092437 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:21.092443 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:21.092593 (+ 150us) client_negotiation.cc:769] Sending connection context
0812 16:37:21.092686 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 16:37:21.092812 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":117,"threads_started":1}
I20260812 16:37:21.093555 8630 heartbeater.cc:347] Connected to a master server at 127.0.253.254:35777
I20260812 16:37:21.093662 8630 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:21.093889 8630 heartbeater.cc:511] Master 127.0.253.254:35777 requested a full tablet report, sending...
I20260812 16:37:21.094304 7817 ts_manager.cc:194] Registered new tserver with Master: f9ca086b5d43407982fc8dcb8987acdd (127.0.253.195:39263)
I20260812 16:37:21.094698 7817 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.195@KRBTEST.COM'} at 127.0.253.195:40543
I20260812 16:37:21.100764 1015 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 16:37:21 dist-test-slave-w4v5 krb5kdc[7749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 16:37:21 dist-test-slave-w4v5 krb5kdc[7749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.253.193@KRBTEST.COM
I20260812 16:37:21.125909 8639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:21.117933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58104 (local address 127.0.253.193:44123)
0812 16:37:21.118126 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:21.118132 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:21.118144 (+ 12us) server_negotiation.cc:407] Connection header received
0812 16:37:21.118174 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:21.118176 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:21.118191 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:21.118261 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:21.118832 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:21.119671 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:21.120434 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:21.120792 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:21.120852 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:21.123266 (+ 2414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:21.123275 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:21.123275 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:21.123282 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:21.124699 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:21.125046 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:21.125063 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:21.125063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:21.125144 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:21.125349 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:21.125349 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:21.125350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:21.125494 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:21.125604 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:21.125689 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 16:37:21.125759 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":96,"threads_started":1}
Aug 12 16:37:21 dist-test-slave-w4v5 krb5kdc[7749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:21.134053 7880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:21.127448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49772 (local address 127.0.253.254:35777)
0812 16:37:21.127496 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:21.127498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:21.127514 (+ 16us) server_negotiation.cc:407] Connection header received
0812 16:37:21.127556 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:21.127556 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:21.127565 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:21.127609 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:21.128045 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:21.128523 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:21.129254 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:21.129439 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:21.129492 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:21.131682 (+ 2190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:21.131684 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:21.131684 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:21.131690 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:21.132905 (+ 1215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:21.133238 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:21.133239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:21.133239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:21.133291 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:21.133469 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:21.133470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:21.133470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:21.133613 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:21.133743 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:21.133837 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 16:37:21.133912 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 16:37:21.134289 7817 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49772
I20260812 16:37:21.134645 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 7911
I20260812 16:37:21.154697 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 8156
I20260812 16:37:21.168905 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 8410
I20260812 16:37:21.179663 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 7771
2026-08-12T16:37:21Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2199 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:37:21 dist-test-slave-w4v5 krb5kdc[8648](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:37:21 dist-test-slave-w4v5 krb5kdc[8648](info): set up 2 sockets
Aug 12 16:37:21 dist-test-slave-w4v5 krb5kdc[8648](info): commencing operation
krb5kdc: starting...
W20260812 16:37:22.504172 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.295s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:37:22 dist-test-slave-w4v5 krb5kdc[8648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:37:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:37:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
I20260812 16:37:22.632045 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:41207
--webserver_interface=127.0.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:38845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:41207
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:22.716365 8671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:22.716570 8671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:22.716593 8671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:22.717746 8671 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:37:22.717789 8671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:22.717803 8671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:22.717815 8671 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:37:22.717828 8671 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:37:22.719372 8671 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:38845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:41207
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:41207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:22.719630 8671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:22.719869 8671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:22.724803 8686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:22.724823 8683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:22.724853 8671 server_base.cc:1034] running on GCE node
W20260812 16:37:22.724864 8684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:22.725278 8671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:22.725574 8671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:22.726742 8671 hybrid_clock.cc:648] HybridClock initialized: now 1786552642726722 us; error 37 us; skew 500 ppm
Aug 12 16:37:22 dist-test-slave-w4v5 krb5kdc[8648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552642, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:22.729312 8671 init.cc:377] Logged in from keytab as kudu/127.0.253.254@KRBTEST.COM (short username kudu)
I20260812 16:37:22.729713 8671 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:22.729784 8671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:22.729938 8671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:22.731012 8671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "1627ae4a6a764c6b908af4ed8d1e5a98"
format_stamp: "Formatted at 2026-08-12 16:37:22 on dist-test-slave-w4v5"
server_key: "23f05ddca27bc7aee7eb19eccb193c25"
server_key_iv: "efbe9dafcef8a3525874724bee32f4ba"
server_key_version: "encryptionkey@0"
I20260812 16:37:22.731453 8671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "1627ae4a6a764c6b908af4ed8d1e5a98"
format_stamp: "Formatted at 2026-08-12 16:37:22 on dist-test-slave-w4v5"
server_key: "23f05ddca27bc7aee7eb19eccb193c25"
server_key_iv: "efbe9dafcef8a3525874724bee32f4ba"
server_key_version: "encryptionkey@0"
I20260812 16:37:22.732769 8671 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:22.733484 8696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:22.733645 8671 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:37:22.733749 8671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "1627ae4a6a764c6b908af4ed8d1e5a98"
format_stamp: "Formatted at 2026-08-12 16:37:22 on dist-test-slave-w4v5"
server_key: "23f05ddca27bc7aee7eb19eccb193c25"
server_key_iv: "efbe9dafcef8a3525874724bee32f4ba"
server_key_version: "encryptionkey@0"
I20260812 16:37:22.733826 8671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:22.757238 8671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:22.760368 8671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:22.760541 8671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:22.765668 8671 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:41207
I20260812 16:37:22.765688 8778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:41207 every 8 connection(s)
I20260812 16:37:22.766000 8671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:37:22.767282 8780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:22.768092 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 8671
I20260812 16:37:22.768221 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:37:22.768415 1015 external_mini_cluster.cc:1428] Setting key 09da77f68851ed84cdc133c6e133160f
I20260812 16:37:22.770704 8780 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1627ae4a6a764c6b908af4ed8d1e5a98: Bootstrap starting.
I20260812 16:37:22.771413 8780 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1627ae4a6a764c6b908af4ed8d1e5a98: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:22.771718 8780 log.cc:824] T 00000000000000000000000000000000 P 1627ae4a6a764c6b908af4ed8d1e5a98: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:22.772535 8780 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1627ae4a6a764c6b908af4ed8d1e5a98: No bootstrap required, opened a new log
I20260812 16:37:22.775194 8780 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1627ae4a6a764c6b908af4ed8d1e5a98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1627ae4a6a764c6b908af4ed8d1e5a98" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 41207 } }
I20260812 16:37:22.775372 8780 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1627ae4a6a764c6b908af4ed8d1e5a98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:22.775416 8780 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1627ae4a6a764c6b908af4ed8d1e5a98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1627ae4a6a764c6b908af4ed8d1e5a98, State: Initialized, Role: FOLLOWER
I20260812 16:37:22.775578 8780 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1627ae4a6a764c6b908af4ed8d1e5a98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1627ae4a6a764c6b908af4ed8d1e5a98" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 41207 } }
I20260812 16:37:22.775660 8780 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1627ae4a6a764c6b908af4ed8d1e5a98 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:37:22.775683 8780 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1627ae4a6a764c6b908af4ed8d1e5a98 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:37:22.775746 8780 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1627ae4a6a764c6b908af4ed8d1e5a98 [term 0 FOLLOWER]: Advancing to term 1
Aug 12 16:37:22 dist-test-slave-w4v5 krb5kdc[8648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:22.776404 8780 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1627ae4a6a764c6b908af4ed8d1e5a98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1627ae4a6a764c6b908af4ed8d1e5a98" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 41207 } }
I20260812 16:37:22.776548 8780 leader_election.cc:302] T 00000000000000000000000000000000 P 1627ae4a6a764c6b908af4ed8d1e5a98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1627ae4a6a764c6b908af4ed8d1e5a98; no voters:
I20260812 16:37:22.776757 8780 leader_election.cc:288] T 00000000000000000000000000000000 P 1627ae4a6a764c6b908af4ed8d1e5a98 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:37:22.776834 8791 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1627ae4a6a764c6b908af4ed8d1e5a98 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:37:22.777037 8791 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1627ae4a6a764c6b908af4ed8d1e5a98 [term 1 LEADER]: Becoming Leader. State: Replica: 1627ae4a6a764c6b908af4ed8d1e5a98, State: Running, Role: LEADER
I20260812 16:37:22.777076 8780 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1627ae4a6a764c6b908af4ed8d1e5a98 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:37:22.777236 8791 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1627ae4a6a764c6b908af4ed8d1e5a98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1627ae4a6a764c6b908af4ed8d1e5a98" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 41207 } }
I20260812 16:37:22.777817 8793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1627ae4a6a764c6b908af4ed8d1e5a98 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1627ae4a6a764c6b908af4ed8d1e5a98. Latest consensus state: current_term: 1 leader_uuid: "1627ae4a6a764c6b908af4ed8d1e5a98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1627ae4a6a764c6b908af4ed8d1e5a98" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 41207 } } }
I20260812 16:37:22.777915 8793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1627ae4a6a764c6b908af4ed8d1e5a98 [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:22.778157 8807 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:37:22.778575 8792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1627ae4a6a764c6b908af4ed8d1e5a98 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1627ae4a6a764c6b908af4ed8d1e5a98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1627ae4a6a764c6b908af4ed8d1e5a98" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 41207 } } }
I20260812 16:37:22.778667 8792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1627ae4a6a764c6b908af4ed8d1e5a98 [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:22.778873 8807 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:37:22.781345 8786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:22.768995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38550 (local address 127.0.253.254:41207)
0812 16:37:22.769170 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:22.769179 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:22.769206 (+ 27us) server_negotiation.cc:407] Connection header received
0812 16:37:22.769612 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:22.769618 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:22.769666 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:22.769783 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:22.770626 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:22.771703 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:22.772736 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:22.773153 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:22.773234 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:22.776598 (+ 3364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:22.776603 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:22.776603 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:22.776617 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:22.779553 (+ 2936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:22.779968 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:22.779972 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:22.779972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:22.780074 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:22.780287 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:22.780288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:22.780288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:22.780595 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:22.780757 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:22.780995 (+ 238us) server_negotiation.cc:299] Negotiation successful
0812 16:37:22.781141 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":85,"threads_started":1}
I20260812 16:37:22.783584 8807 catalog_manager.cc:1347] Generated new cluster ID: f8e23d4b49324c5082dd43b717d34d97
I20260812 16:37:22.783656 8807 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:37:22.801748 8807 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:37:22.802557 8807 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:37:22.806316 8807 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1627ae4a6a764c6b908af4ed8d1e5a98: Generated new TSK 0
I20260812 16:37:22.806515 8807 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab.
Entry for principal kudu/127.0.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab.
I20260812 16:37:22.841895 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.193:0
--local_ip_for_outbound_sockets=127.0.253.193
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:41207
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:38845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:22.927695 8824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:22.927980 8824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:22.928028 8824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:22.929256 8824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:22.929337 8824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:22.929384 8824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.193
I20260812 16:37:22.931011 8824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:38845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:41207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.193
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:22.931308 8824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:22.931548 8824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:22.936807 8832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:22.936811 8831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:22.936849 8824 server_base.cc:1034] running on GCE node
W20260812 16:37:22.936820 8836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:22.937228 8824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:22.937539 8824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:22.938683 8824 hybrid_clock.cc:648] HybridClock initialized: now 1786552642938657 us; error 38 us; skew 500 ppm
Aug 12 16:37:22 dist-test-slave-w4v5 krb5kdc[8648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552642, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:22.941367 8824 init.cc:377] Logged in from keytab as kudu/127.0.253.193@KRBTEST.COM (short username kudu)
I20260812 16:37:22.941766 8824 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:22.941814 8824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:22.941944 8824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:22.943027 8824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-0/data/instance:
uuid: "27bd56c664f5442f801d24e4cbce13f8"
format_stamp: "Formatted at 2026-08-12 16:37:22 on dist-test-slave-w4v5"
server_key: "e9e91e042fd475734139ca501f3df013"
server_key_iv: "50291a1f483c92c50554e19def83a9b8"
server_key_version: "encryptionkey@0"
I20260812 16:37:22.943450 8824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance:
uuid: "27bd56c664f5442f801d24e4cbce13f8"
format_stamp: "Formatted at 2026-08-12 16:37:22 on dist-test-slave-w4v5"
server_key: "e9e91e042fd475734139ca501f3df013"
server_key_iv: "50291a1f483c92c50554e19def83a9b8"
server_key_version: "encryptionkey@0"
I20260812 16:37:22.944767 8824 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:22.945513 8844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:22.945720 8824 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:22.945816 8824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-0/wal
uuid: "27bd56c664f5442f801d24e4cbce13f8"
format_stamp: "Formatted at 2026-08-12 16:37:22 on dist-test-slave-w4v5"
server_key: "e9e91e042fd475734139ca501f3df013"
server_key_iv: "50291a1f483c92c50554e19def83a9b8"
server_key_version: "encryptionkey@0"
I20260812 16:37:22.945899 8824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:22.963407 8824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:22.966235 8824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:22.966392 8824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:22.966687 8824 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:37:22.967051 8824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:22.967132 8824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:22.967190 8824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:22.967217 8824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:22.973371 8824 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.193:33889
I20260812 16:37:22.973413 9014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.193:33889 every 8 connection(s)
I20260812 16:37:22.973701 8824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
I20260812 16:37:22.978621 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 8824
I20260812 16:37:22.978751 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance
I20260812 16:37:22.978966 1015 external_mini_cluster.cc:1428] Setting key c3c3342e05fe5f596b13e07a3517da39
Aug 12 16:37:22 dist-test-slave-w4v5 krb5kdc[8648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552642, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:22.984977 9018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:22.974359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41207 (local address 127.0.253.193:55365)
0812 16:37:22.974532 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:22.974577 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:22.975119 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:22.975397 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:22.975401 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:22.975666 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:22.976235 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:22.976242 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:22.976975 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:22.976976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:22.977927 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:22.977930 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:22.978496 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:22.978497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:22.978656 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:22.979416 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:22.979428 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:22.981120 (+ 1692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:22.983167 (+ 2047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:22.983177 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:22.983180 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:22.983662 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:22.983982 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:22.983983 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:22.983983 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:22.984112 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:22.984421 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:22.984427 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:22.984557 (+ 130us) client_negotiation.cc:769] Sending connection context
0812 16:37:22.984648 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 16:37:22.984801 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":93,"threads_started":1}
I20260812 16:37:22.984967 8786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:22.974405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:55365 (local address 127.0.253.254:41207)
0812 16:37:22.974460 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:22.974462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:22.974654 (+ 192us) server_negotiation.cc:407] Connection header received
0812 16:37:22.975243 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:22.975245 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:22.975257 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:22.975324 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:22.976333 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:22.976886 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:22.978029 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:22.978378 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:22.978479 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:22.981229 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:22.981231 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:22.981232 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:22.981242 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:22.983006 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:22.983788 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:22.983789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:22.983790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:22.983887 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:22.984176 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:22.984177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:22.984178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:22.984352 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:22.984428 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:22.984700 (+ 272us) server_negotiation.cc:299] Negotiation successful
0812 16:37:22.984820 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 16:37:22.985867 9016 heartbeater.cc:347] Connected to a master server at 127.0.253.254:41207
I20260812 16:37:22.985985 9016 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:22.986145 9016 heartbeater.cc:511] Master 127.0.253.254:41207 requested a full tablet report, sending...
I20260812 16:37:22.986701 8721 ts_manager.cc:194] Registered new tserver with Master: 27bd56c664f5442f801d24e4cbce13f8 (127.0.253.193:33889)
I20260812 16:37:22.987387 8721 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.193@KRBTEST.COM'} at 127.0.253.193:55365
WARNING: no policy specified for kudu/127.0.253.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab.
Entry for principal kudu/127.0.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab.
I20260812 16:37:23.009820 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.194:0
--local_ip_for_outbound_sockets=127.0.253.194
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:41207
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:38845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:23.094167 9022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:23.094388 9022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:23.094413 9022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:23.095633 9022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:23.095676 9022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:23.095703 9022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.194
I20260812 16:37:23.097272 9022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:38845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:41207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.194
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:23.097537 9022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:23.097777 9022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 16:37:23.102888 9022 server_base.cc:1034] running on GCE node
W20260812 16:37:23.102859 9029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:23.102859 9030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:23.102859 9033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:23.103410 9022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:23.103665 9022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:23.104843 9022 hybrid_clock.cc:648] HybridClock initialized: now 1786552643104827 us; error 30 us; skew 500 ppm
Aug 12 16:37:23 dist-test-slave-w4v5 krb5kdc[8648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552643, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:23.107653 9022 init.cc:377] Logged in from keytab as kudu/127.0.253.194@KRBTEST.COM (short username kudu)
I20260812 16:37:23.108021 9022 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:23.108094 9022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:23.108249 9022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:23.109376 9022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-1/data/instance:
uuid: "7a011299d20d4ebfa87007fc0ef36f46"
format_stamp: "Formatted at 2026-08-12 16:37:23 on dist-test-slave-w4v5"
server_key: "3493c1a42ceba6efcb23b516a4a0d7c8"
server_key_iv: "4875a83359df0da14469bd832fa62df6"
server_key_version: "encryptionkey@0"
I20260812 16:37:23.109745 9022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance:
uuid: "7a011299d20d4ebfa87007fc0ef36f46"
format_stamp: "Formatted at 2026-08-12 16:37:23 on dist-test-slave-w4v5"
server_key: "3493c1a42ceba6efcb23b516a4a0d7c8"
server_key_iv: "4875a83359df0da14469bd832fa62df6"
server_key_version: "encryptionkey@0"
I20260812 16:37:23.111280 9022 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 16:37:23.112134 9043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:23.112325 9022 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:23.112445 9022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-1/wal
uuid: "7a011299d20d4ebfa87007fc0ef36f46"
format_stamp: "Formatted at 2026-08-12 16:37:23 on dist-test-slave-w4v5"
server_key: "3493c1a42ceba6efcb23b516a4a0d7c8"
server_key_iv: "4875a83359df0da14469bd832fa62df6"
server_key_version: "encryptionkey@0"
I20260812 16:37:23.112525 9022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:23.137738 9022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:23.140070 9022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:23.140189 9022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:23.140393 9022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:37:23.140729 9022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:23.140767 9022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:23.140790 9022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:23.140803 9022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:23.146720 9022 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.194:38219
I20260812 16:37:23.146756 9239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.194:38219 every 8 connection(s)
I20260812 16:37:23.147040 9022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
Aug 12 16:37:23 dist-test-slave-w4v5 krb5kdc[8648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552643, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:23.156589 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 9022
I20260812 16:37:23.156752 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance
I20260812 16:37:23.156962 1015 external_mini_cluster.cc:1428] Setting key 1eb9eb8e06c18cc5e1099f3c8e8afde2
I20260812 16:37:23.157081 8786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:23.147896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:50667 (local address 127.0.253.254:41207)
0812 16:37:23.147957 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:23.147959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:23.148152 (+ 193us) server_negotiation.cc:407] Connection header received
0812 16:37:23.148666 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:23.148667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:23.148679 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:23.148754 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:23.149831 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:23.150417 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:23.151622 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:23.151823 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:23.151891 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:23.154123 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:23.154125 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:23.154125 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:23.154132 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:23.155497 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:23.155894 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:23.155895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:23.155896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:23.155952 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:23.156233 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:23.156234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:23.156234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:23.156341 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:23.156396 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:23.156892 (+ 496us) server_negotiation.cc:299] Negotiation successful
0812 16:37:23.156972 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 16:37:23.157330 9247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:23.147816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41207 (local address 127.0.253.194:50667)
0812 16:37:23.147990 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:23.148035 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:23.148560 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:23.148865 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:23.148869 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:23.149242 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:23.149736 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:23.149742 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:23.150576 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:23.150577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:23.151519 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:23.151522 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:23.151910 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:23.151910 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:23.152025 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:23.152608 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:23.152617 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:23.154026 (+ 1409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:23.155568 (+ 1542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:23.155607 (+ 39us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:23.155609 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:23.155829 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:23.156041 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:23.156042 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:23.156042 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:23.156166 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:23.156513 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:23.156522 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:23.156699 (+ 177us) client_negotiation.cc:769] Sending connection context
0812 16:37:23.156848 (+ 149us) client_negotiation.cc:240] Negotiation successful
0812 16:37:23.157038 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":98,"threads_started":1}
I20260812 16:37:23.158007 9242 heartbeater.cc:347] Connected to a master server at 127.0.253.254:41207
I20260812 16:37:23.158130 9242 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:23.158289 9242 heartbeater.cc:511] Master 127.0.253.254:41207 requested a full tablet report, sending...
I20260812 16:37:23.158684 8721 ts_manager.cc:194] Registered new tserver with Master: 7a011299d20d4ebfa87007fc0ef36f46 (127.0.253.194:38219)
I20260812 16:37:23.159050 8721 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.194@KRBTEST.COM'} at 127.0.253.194:50667
WARNING: no policy specified for kudu/127.0.253.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab.
Entry for principal kudu/127.0.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab.
I20260812 16:37:23.186013 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.195:0
--local_ip_for_outbound_sockets=127.0.253.195
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:41207
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:38845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:23.272591 9252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:23.272812 9252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:23.272838 9252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:23.274062 9252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:23.274109 9252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:23.274138 9252 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.195
I20260812 16:37:23.275784 9252 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:38845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:41207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.195
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:23.276037 9252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:23.276285 9252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:23.281942 9263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:23.281939 9272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:23.282025 9268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:23.282172 9252 server_base.cc:1034] running on GCE node
I20260812 16:37:23.282436 9252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:23.282699 9252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:23.283897 9252 hybrid_clock.cc:648] HybridClock initialized: now 1786552643283878 us; error 46 us; skew 500 ppm
Aug 12 16:37:23 dist-test-slave-w4v5 krb5kdc[8648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552643, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:23.286562 9252 init.cc:377] Logged in from keytab as kudu/127.0.253.195@KRBTEST.COM (short username kudu)
I20260812 16:37:23.286953 9252 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:23.287004 9252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:23.287173 9252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:23.288251 9252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-2/data/instance:
uuid: "a10d2415c1f14d0bbf3386dbcdc80698"
format_stamp: "Formatted at 2026-08-12 16:37:23 on dist-test-slave-w4v5"
server_key: "eb21ce3f9d6d4bbfbeaacee8679434b9"
server_key_iv: "1f629bf5cdf1fa2d7e6dfa96cbda04f9"
server_key_version: "encryptionkey@0"
I20260812 16:37:23.288566 9252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance:
uuid: "a10d2415c1f14d0bbf3386dbcdc80698"
format_stamp: "Formatted at 2026-08-12 16:37:23 on dist-test-slave-w4v5"
server_key: "eb21ce3f9d6d4bbfbeaacee8679434b9"
server_key_iv: "1f629bf5cdf1fa2d7e6dfa96cbda04f9"
server_key_version: "encryptionkey@0"
I20260812 16:37:23.289873 9252 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:37:23.290709 9284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:23.290913 9252 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:23.291002 9252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-2/wal
uuid: "a10d2415c1f14d0bbf3386dbcdc80698"
format_stamp: "Formatted at 2026-08-12 16:37:23 on dist-test-slave-w4v5"
server_key: "eb21ce3f9d6d4bbfbeaacee8679434b9"
server_key_iv: "1f629bf5cdf1fa2d7e6dfa96cbda04f9"
server_key_version: "encryptionkey@0"
I20260812 16:37:23.291059 9252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:23.307260 9252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:23.309224 9252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:23.309340 9252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:23.309549 9252 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:37:23.309827 9252 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:23.309857 9252 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:23.309880 9252 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:23.309892 9252 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:23.316061 9252 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.195:33883
I20260812 16:37:23.316097 9517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.195:33883 every 8 connection(s)
I20260812 16:37:23.316385 9252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
I20260812 16:37:23.321830 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 9252
I20260812 16:37:23.321960 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance
I20260812 16:37:23.322184 1015 external_mini_cluster.cc:1428] Setting key c10be415b74761959480e4c24dbe1e93
Aug 12 16:37:23 dist-test-slave-w4v5 krb5kdc[8648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552643, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:23.325881 8786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:23.317138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:53309 (local address 127.0.253.254:41207)
0812 16:37:23.317217 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:23.317219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:23.317425 (+ 206us) server_negotiation.cc:407] Connection header received
0812 16:37:23.317941 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:23.317942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:23.317954 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:23.318019 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:23.318941 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:23.319520 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:23.320490 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:23.320691 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:23.320759 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:23.323024 (+ 2265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:23.323026 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:23.323026 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:23.323034 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:23.324531 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:23.324972 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:23.324973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:23.324974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:23.325027 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:23.325297 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:23.325298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:23.325298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:23.325407 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:23.325461 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:23.325738 (+ 277us) server_negotiation.cc:299] Negotiation successful
0812 16:37:23.325814 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 16:37:23.326020 9523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:23.317122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41207 (local address 127.0.253.195:53309)
0812 16:37:23.317295 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:23.317337 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:23.317828 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:23.318076 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:23.318079 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:23.318365 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:23.318850 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:23.318856 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:23.319597 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:23.319598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:23.320410 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:23.320412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:23.320795 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:23.320796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:23.320907 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:23.321365 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:23.321373 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:23.322942 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:23.324618 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:23.324625 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:23.324627 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:23.324898 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:23.325105 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:23.325106 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:23.325106 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:23.325236 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:23.325464 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:23.325469 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:23.325594 (+ 125us) client_negotiation.cc:769] Sending connection context
0812 16:37:23.325680 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 16:37:23.325822 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":101,"threads_started":1}
I20260812 16:37:23.326575 9518 heartbeater.cc:347] Connected to a master server at 127.0.253.254:41207
I20260812 16:37:23.326687 9518 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:23.326877 9518 heartbeater.cc:511] Master 127.0.253.254:41207 requested a full tablet report, sending...
I20260812 16:37:23.327369 8721 ts_manager.cc:194] Registered new tserver with Master: a10d2415c1f14d0bbf3386dbcdc80698 (127.0.253.195:33883)
I20260812 16:37:23.327734 8721 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.195@KRBTEST.COM'} at 127.0.253.195:53309
I20260812 16:37:23.334132 1015 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:37:23.345192 8786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:23.335720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38562 (local address 127.0.253.254:41207)
0812 16:37:23.335790 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:23.335793 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:23.335931 (+ 138us) server_negotiation.cc:407] Connection header received
0812 16:37:23.335980 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:23.335981 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:23.335992 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:23.336067 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:23.339586 (+ 3519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:23.340315 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:23.341200 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:23.341383 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:23.341441 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:23.342668 (+ 1227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:23.342669 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:23.342670 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:23.342677 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:23.344242 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:23.344573 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:23.344574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:23.344574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:23.344623 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:23.344767 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:23.344767 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:23.344768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:23.344903 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:23.344980 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:23.345071 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 16:37:23.345127 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 16:37:23.346753 8721 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38562:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:37:23.348439 8721 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:37:23.355540 9546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:23.351410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:33889 (local address 127.0.0.1:45928)
0812 16:37:23.351584 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:23.351626 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:23.351711 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:23.352041 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:23.352044 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:23.352045 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:23.352139 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:23.352141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:23.353542 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:23.353543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:23.354407 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:23.354410 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:23.355216 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:23.355217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:23.355307 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:23.355322 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 16:37:23.355402 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 16:37:23.355455 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":51,"thread_start_us":83,"threads_started":1}
I20260812 16:37:23.355556 9549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:23.351541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45928 (local address 127.0.253.193:33889)
0812 16:37:23.351879 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:23.351885 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:23.351897 (+ 12us) server_negotiation.cc:407] Connection header received
0812 16:37:23.351941 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:23.351943 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:23.351962 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:23.352040 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:23.352244 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:23.353421 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:23.354499 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:23.355131 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:23.355213 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:23.355271 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:23.355426 (+ 155us) server_negotiation.cc:299] Negotiation successful
0812 16:37:23.355468 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":99,"threads_started":1}
I20260812 16:37:23.356112 9553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:23.351885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53450 (local address 127.0.253.194:38219)
0812 16:37:23.352052 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:23.352056 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:23.352067 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:37:23.352108 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:23.352110 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:23.352127 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:23.352220 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:23.352481 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:23.353189 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:23.354847 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:23.355550 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:23.355812 (+ 262us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:23.355877 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:23.355987 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 16:37:23.356026 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":102,"threads_started":1}
I20260812 16:37:23.356127 9550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:23.351622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:38219 (local address 127.0.0.1:53450)
0812 16:37:23.351863 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:23.351875 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:23.351941 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:23.352326 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:23.352327 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:23.352329 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:23.352394 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:23.352396 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:23.353290 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:23.353291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:23.354788 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:23.354790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:23.355813 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:23.355814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:23.355886 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:23.355891 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:37:23.355971 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 16:37:23.356014 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":82,"threads_started":1}
I20260812 16:37:23.356127 9547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:23.351411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.195:33883 (local address 127.0.0.1:39114)
0812 16:37:23.351739 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:23.351750 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:23.351832 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:23.352377 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:23.352378 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:23.352379 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:23.352427 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:23.352428 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:23.353623 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:23.353624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:23.354727 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:23.354730 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:23.355868 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:23.355869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:23.355941 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:23.355946 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:37:23.355983 (+ 37us) client_negotiation.cc:240] Negotiation successful
0812 16:37:23.356026 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"mutex_wait_us":2,"thread_start_us":45,"threads_started":1}
I20260812 16:37:23.356467 9548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:23.351526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39114 (local address 127.0.253.195:33883)
0812 16:37:23.352194 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:23.352198 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:23.352208 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:37:23.352246 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:23.352248 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:23.352264 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:23.352375 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:23.352492 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:23.353527 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:23.354811 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:23.355552 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:23.356256 (+ 704us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:23.356320 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:23.356348 (+ 28us) server_negotiation.cc:299] Negotiation successful
0812 16:37:23.356389 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":98,"threads_started":1}
I20260812 16:37:23.356349 8920 tablet_service.cc:1467] Processing CreateTablet for tablet ac33e7af9254487fb2ed25b611900057 (DEFAULT_TABLE table=test-table [id=67444bb2026248d982627d0dde10c0ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:23.356772 8920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac33e7af9254487fb2ed25b611900057. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:23.357244 9132 tablet_service.cc:1467] Processing CreateTablet for tablet ac33e7af9254487fb2ed25b611900057 (DEFAULT_TABLE table=test-table [id=67444bb2026248d982627d0dde10c0ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:23.357295 9372 tablet_service.cc:1467] Processing CreateTablet for tablet ac33e7af9254487fb2ed25b611900057 (DEFAULT_TABLE table=test-table [id=67444bb2026248d982627d0dde10c0ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:23.357573 9132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac33e7af9254487fb2ed25b611900057. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:23.357625 9372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac33e7af9254487fb2ed25b611900057. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:23.360280 9554 tablet_bootstrap.cc:492] T ac33e7af9254487fb2ed25b611900057 P 7a011299d20d4ebfa87007fc0ef36f46: Bootstrap starting.
I20260812 16:37:23.360669 9555 tablet_bootstrap.cc:492] T ac33e7af9254487fb2ed25b611900057 P 27bd56c664f5442f801d24e4cbce13f8: Bootstrap starting.
I20260812 16:37:23.361014 9554 tablet_bootstrap.cc:654] T ac33e7af9254487fb2ed25b611900057 P 7a011299d20d4ebfa87007fc0ef36f46: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:23.360996 9559 tablet_bootstrap.cc:492] T ac33e7af9254487fb2ed25b611900057 P a10d2415c1f14d0bbf3386dbcdc80698: Bootstrap starting.
I20260812 16:37:23.361316 9555 tablet_bootstrap.cc:654] T ac33e7af9254487fb2ed25b611900057 P 27bd56c664f5442f801d24e4cbce13f8: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:23.361804 9554 log.cc:824] T ac33e7af9254487fb2ed25b611900057 P 7a011299d20d4ebfa87007fc0ef36f46: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:23.361857 9559 tablet_bootstrap.cc:654] T ac33e7af9254487fb2ed25b611900057 P a10d2415c1f14d0bbf3386dbcdc80698: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:23.361840 9555 log.cc:824] T ac33e7af9254487fb2ed25b611900057 P 27bd56c664f5442f801d24e4cbce13f8: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:23.362468 9559 log.cc:824] T ac33e7af9254487fb2ed25b611900057 P a10d2415c1f14d0bbf3386dbcdc80698: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:23.362634 9554 tablet_bootstrap.cc:492] T ac33e7af9254487fb2ed25b611900057 P 7a011299d20d4ebfa87007fc0ef36f46: No bootstrap required, opened a new log
I20260812 16:37:23.362632 9555 tablet_bootstrap.cc:492] T ac33e7af9254487fb2ed25b611900057 P 27bd56c664f5442f801d24e4cbce13f8: No bootstrap required, opened a new log
I20260812 16:37:23.362713 9554 ts_tablet_manager.cc:1397] T ac33e7af9254487fb2ed25b611900057 P 7a011299d20d4ebfa87007fc0ef36f46: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:37:23.362707 9555 ts_tablet_manager.cc:1397] T ac33e7af9254487fb2ed25b611900057 P 27bd56c664f5442f801d24e4cbce13f8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:37:23.363253 9559 tablet_bootstrap.cc:492] T ac33e7af9254487fb2ed25b611900057 P a10d2415c1f14d0bbf3386dbcdc80698: No bootstrap required, opened a new log
I20260812 16:37:23.363334 9559 ts_tablet_manager.cc:1397] T ac33e7af9254487fb2ed25b611900057 P a10d2415c1f14d0bbf3386dbcdc80698: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:37:23.364650 9559 raft_consensus.cc:348] T ac33e7af9254487fb2ed25b611900057 P a10d2415c1f14d0bbf3386dbcdc80698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10d2415c1f14d0bbf3386dbcdc80698" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 33883 } } peers { permanent_uuid: "27bd56c664f5442f801d24e4cbce13f8" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 33889 } } peers { permanent_uuid: "7a011299d20d4ebfa87007fc0ef36f46" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38219 } }
I20260812 16:37:23.364815 9559 raft_consensus.cc:374] T ac33e7af9254487fb2ed25b611900057 P a10d2415c1f14d0bbf3386dbcdc80698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:23.364742 9554 raft_consensus.cc:348] T ac33e7af9254487fb2ed25b611900057 P 7a011299d20d4ebfa87007fc0ef36f46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10d2415c1f14d0bbf3386dbcdc80698" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 33883 } } peers { permanent_uuid: "27bd56c664f5442f801d24e4cbce13f8" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 33889 } } peers { permanent_uuid: "7a011299d20d4ebfa87007fc0ef36f46" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38219 } }
I20260812 16:37:23.364742 9555 raft_consensus.cc:348] T ac33e7af9254487fb2ed25b611900057 P 27bd56c664f5442f801d24e4cbce13f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10d2415c1f14d0bbf3386dbcdc80698" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 33883 } } peers { permanent_uuid: "27bd56c664f5442f801d24e4cbce13f8" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 33889 } } peers { permanent_uuid: "7a011299d20d4ebfa87007fc0ef36f46" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38219 } }
I20260812 16:37:23.364861 9559 raft_consensus.cc:729] T ac33e7af9254487fb2ed25b611900057 P a10d2415c1f14d0bbf3386dbcdc80698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a10d2415c1f14d0bbf3386dbcdc80698, State: Initialized, Role: FOLLOWER
I20260812 16:37:23.364892 9554 raft_consensus.cc:374] T ac33e7af9254487fb2ed25b611900057 P 7a011299d20d4ebfa87007fc0ef36f46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:23.364892 9555 raft_consensus.cc:374] T ac33e7af9254487fb2ed25b611900057 P 27bd56c664f5442f801d24e4cbce13f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:23.364938 9555 raft_consensus.cc:729] T ac33e7af9254487fb2ed25b611900057 P 27bd56c664f5442f801d24e4cbce13f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27bd56c664f5442f801d24e4cbce13f8, State: Initialized, Role: FOLLOWER
I20260812 16:37:23.364938 9554 raft_consensus.cc:729] T ac33e7af9254487fb2ed25b611900057 P 7a011299d20d4ebfa87007fc0ef36f46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a011299d20d4ebfa87007fc0ef36f46, State: Initialized, Role: FOLLOWER
I20260812 16:37:23.364975 9559 consensus_queue.cc:260] T ac33e7af9254487fb2ed25b611900057 P a10d2415c1f14d0bbf3386dbcdc80698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10d2415c1f14d0bbf3386dbcdc80698" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 33883 } } peers { permanent_uuid: "27bd56c664f5442f801d24e4cbce13f8" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 33889 } } peers { permanent_uuid: "7a011299d20d4ebfa87007fc0ef36f46" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38219 } }
I20260812 16:37:23.365044 9554 consensus_queue.cc:260] T ac33e7af9254487fb2ed25b611900057 P 7a011299d20d4ebfa87007fc0ef36f46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10d2415c1f14d0bbf3386dbcdc80698" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 33883 } } peers { permanent_uuid: "27bd56c664f5442f801d24e4cbce13f8" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 33889 } } peers { permanent_uuid: "7a011299d20d4ebfa87007fc0ef36f46" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38219 } }
I20260812 16:37:23.365062 9555 consensus_queue.cc:260] T ac33e7af9254487fb2ed25b611900057 P 27bd56c664f5442f801d24e4cbce13f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10d2415c1f14d0bbf3386dbcdc80698" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 33883 } } peers { permanent_uuid: "27bd56c664f5442f801d24e4cbce13f8" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 33889 } } peers { permanent_uuid: "7a011299d20d4ebfa87007fc0ef36f46" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38219 } }
I20260812 16:37:23.365269 9559 ts_tablet_manager.cc:1428] T ac33e7af9254487fb2ed25b611900057 P a10d2415c1f14d0bbf3386dbcdc80698: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:37:23.365278 9518 heartbeater.cc:503] Master 127.0.253.254:41207 was elected leader, sending a full tablet report...
I20260812 16:37:23.365322 9555 ts_tablet_manager.cc:1428] T ac33e7af9254487fb2ed25b611900057 P 27bd56c664f5442f801d24e4cbce13f8: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 16:37:23.365322 9554 ts_tablet_manager.cc:1428] T ac33e7af9254487fb2ed25b611900057 P 7a011299d20d4ebfa87007fc0ef36f46: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 16:37:23.365453 9016 heartbeater.cc:503] Master 127.0.253.254:41207 was elected leader, sending a full tablet report...
I20260812 16:37:23.365705 9242 heartbeater.cc:503] Master 127.0.253.254:41207 was elected leader, sending a full tablet report...
W20260812 16:37:23.397652 9245 tablet.cc:2369] T ac33e7af9254487fb2ed25b611900057 P 7a011299d20d4ebfa87007fc0ef36f46: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:37:23.449939 9564 raft_consensus.cc:482] T ac33e7af9254487fb2ed25b611900057 P 7a011299d20d4ebfa87007fc0ef36f46 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:37:23.450071 9564 raft_consensus.cc:504] T ac33e7af9254487fb2ed25b611900057 P 7a011299d20d4ebfa87007fc0ef36f46 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10d2415c1f14d0bbf3386dbcdc80698" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 33883 } } peers { permanent_uuid: "27bd56c664f5442f801d24e4cbce13f8" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 33889 } } peers { permanent_uuid: "7a011299d20d4ebfa87007fc0ef36f46" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38219 } }
I20260812 16:37:23.450433 9564 leader_election.cc:288] T ac33e7af9254487fb2ed25b611900057 P 7a011299d20d4ebfa87007fc0ef36f46 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a10d2415c1f14d0bbf3386dbcdc80698 (127.0.253.195:33883), 27bd56c664f5442f801d24e4cbce13f8 (127.0.253.193:33889)
I20260812 16:37:23.454373 9549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:23.450699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:54825 (local address 127.0.253.193:33889)
0812 16:37:23.450744 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:23.450746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:23.450756 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:37:23.450842 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:23.450843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:23.450858 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:23.450911 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:23.451104 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:23.451883 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:23.453417 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:23.453971 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:23.454069 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:23.454077 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:23.454258 (+ 181us) server_negotiation.cc:299] Negotiation successful
0812 16:37:23.454297 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 16:37:23.454373 9548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:23.450733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:50879 (local address 127.0.253.195:33883)
0812 16:37:23.450792 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:23.450795 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:23.450944 (+ 149us) server_negotiation.cc:407] Connection header received
0812 16:37:23.451033 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:23.451034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:23.451048 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:23.451166 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:23.451338 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:23.452011 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:23.453417 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:23.454001 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:23.454067 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:23.454077 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:23.454258 (+ 181us) server_negotiation.cc:299] Negotiation successful
0812 16:37:23.454298 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 16:37:23.454396 9247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:23.450644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:33889 (local address 127.0.253.194:54825)
0812 16:37:23.450710 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:23.450723 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:23.450771 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:23.450939 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:23.450941 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:23.450943 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:23.450996 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:23.450998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:23.451981 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:23.451982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:23.453304 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:23.453307 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:23.454087 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:23.454089 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:23.454170 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:23.454175 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:37:23.454240 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 16:37:23.454286 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":6}
I20260812 16:37:23.454402 9566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:23.450649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.195:33883 (local address 127.0.253.194:50879)
0812 16:37:23.450860 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:23.450870 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:23.450959 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:23.451174 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:23.451174 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:23.451176 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:23.451250 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:23.451251 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:23.452110 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:23.452111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:23.453304 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:23.453307 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:23.454087 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:23.454089 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:23.454175 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:23.454177 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 16:37:23.454240 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 16:37:23.454286 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":33,"thread_start_us":125,"threads_started":1}
I20260812 16:37:23.454864 9425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac33e7af9254487fb2ed25b611900057" candidate_uuid: "7a011299d20d4ebfa87007fc0ef36f46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a10d2415c1f14d0bbf3386dbcdc80698" is_pre_election: true
I20260812 16:37:23.454864 8950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac33e7af9254487fb2ed25b611900057" candidate_uuid: "7a011299d20d4ebfa87007fc0ef36f46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27bd56c664f5442f801d24e4cbce13f8" is_pre_election: true
I20260812 16:37:23.455044 9425 raft_consensus.cc:2454] T ac33e7af9254487fb2ed25b611900057 P a10d2415c1f14d0bbf3386dbcdc80698 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a011299d20d4ebfa87007fc0ef36f46 in term 0.
I20260812 16:37:23.455044 8950 raft_consensus.cc:2454] T ac33e7af9254487fb2ed25b611900057 P 27bd56c664f5442f801d24e4cbce13f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a011299d20d4ebfa87007fc0ef36f46 in term 0.
I20260812 16:37:23.455338 9046 leader_election.cc:302] T ac33e7af9254487fb2ed25b611900057 P 7a011299d20d4ebfa87007fc0ef36f46 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a011299d20d4ebfa87007fc0ef36f46, a10d2415c1f14d0bbf3386dbcdc80698; no voters:
I20260812 16:37:23.455564 9564 raft_consensus.cc:2781] T ac33e7af9254487fb2ed25b611900057 P 7a011299d20d4ebfa87007fc0ef36f46 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:37:23.455631 9564 raft_consensus.cc:482] T ac33e7af9254487fb2ed25b611900057 P 7a011299d20d4ebfa87007fc0ef36f46 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:37:23.455665 9564 raft_consensus.cc:3037] T ac33e7af9254487fb2ed25b611900057 P 7a011299d20d4ebfa87007fc0ef36f46 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:23.456368 9564 raft_consensus.cc:504] T ac33e7af9254487fb2ed25b611900057 P 7a011299d20d4ebfa87007fc0ef36f46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10d2415c1f14d0bbf3386dbcdc80698" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 33883 } } peers { permanent_uuid: "27bd56c664f5442f801d24e4cbce13f8" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 33889 } } peers { permanent_uuid: "7a011299d20d4ebfa87007fc0ef36f46" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38219 } }
I20260812 16:37:23.456521 9564 leader_election.cc:288] T ac33e7af9254487fb2ed25b611900057 P 7a011299d20d4ebfa87007fc0ef36f46 [CANDIDATE]: Term 1 election: Requested vote from peers a10d2415c1f14d0bbf3386dbcdc80698 (127.0.253.195:33883), 27bd56c664f5442f801d24e4cbce13f8 (127.0.253.193:33889)
I20260812 16:37:23.456708 9425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac33e7af9254487fb2ed25b611900057" candidate_uuid: "7a011299d20d4ebfa87007fc0ef36f46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a10d2415c1f14d0bbf3386dbcdc80698"
I20260812 16:37:23.456704 8950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac33e7af9254487fb2ed25b611900057" candidate_uuid: "7a011299d20d4ebfa87007fc0ef36f46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27bd56c664f5442f801d24e4cbce13f8"
I20260812 16:37:23.456787 9425 raft_consensus.cc:3037] T ac33e7af9254487fb2ed25b611900057 P a10d2415c1f14d0bbf3386dbcdc80698 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:23.456804 8950 raft_consensus.cc:3037] T ac33e7af9254487fb2ed25b611900057 P 27bd56c664f5442f801d24e4cbce13f8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:23.457520 8950 raft_consensus.cc:2454] T ac33e7af9254487fb2ed25b611900057 P 27bd56c664f5442f801d24e4cbce13f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a011299d20d4ebfa87007fc0ef36f46 in term 1.
I20260812 16:37:23.457543 9425 raft_consensus.cc:2454] T ac33e7af9254487fb2ed25b611900057 P a10d2415c1f14d0bbf3386dbcdc80698 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a011299d20d4ebfa87007fc0ef36f46 in term 1.
I20260812 16:37:23.457705 9045 leader_election.cc:302] T ac33e7af9254487fb2ed25b611900057 P 7a011299d20d4ebfa87007fc0ef36f46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27bd56c664f5442f801d24e4cbce13f8, 7a011299d20d4ebfa87007fc0ef36f46; no voters:
I20260812 16:37:23.457818 9564 raft_consensus.cc:2781] T ac33e7af9254487fb2ed25b611900057 P 7a011299d20d4ebfa87007fc0ef36f46 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:37:23.457988 9564 raft_consensus.cc:686] T ac33e7af9254487fb2ed25b611900057 P 7a011299d20d4ebfa87007fc0ef36f46 [term 1 LEADER]: Becoming Leader. State: Replica: 7a011299d20d4ebfa87007fc0ef36f46, State: Running, Role: LEADER
I20260812 16:37:23.458097 9564 consensus_queue.cc:237] T ac33e7af9254487fb2ed25b611900057 P 7a011299d20d4ebfa87007fc0ef36f46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10d2415c1f14d0bbf3386dbcdc80698" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 33883 } } peers { permanent_uuid: "27bd56c664f5442f801d24e4cbce13f8" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 33889 } } peers { permanent_uuid: "7a011299d20d4ebfa87007fc0ef36f46" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38219 } }
I20260812 16:37:23.458994 8721 catalog_manager.cc:5495] T ac33e7af9254487fb2ed25b611900057 P 7a011299d20d4ebfa87007fc0ef36f46 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a011299d20d4ebfa87007fc0ef36f46 (127.0.253.194). New cstate: current_term: 1 leader_uuid: "7a011299d20d4ebfa87007fc0ef36f46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10d2415c1f14d0bbf3386dbcdc80698" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 33883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27bd56c664f5442f801d24e4cbce13f8" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 33889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a011299d20d4ebfa87007fc0ef36f46" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38219 } health_report { overall_health: HEALTHY } } }
I20260812 16:37:23.471096 9553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:23.468692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53462 (local address 127.0.253.194:38219)
0812 16:37:23.468742 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:23.468744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:23.468753 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:37:23.468806 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:23.468807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:23.468824 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:23.468899 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:37:23.469048 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:23.469515 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:23.470273 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:23.470462 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:23.470538 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:23.470716 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:37:23.470825 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:37:23.470923 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:23.470948 (+ 25us) server_negotiation.cc:299] Negotiation successful
0812 16:37:23.470969 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 16:37:23.472958 9425 raft_consensus.cc:1261] T ac33e7af9254487fb2ed25b611900057 P a10d2415c1f14d0bbf3386dbcdc80698 [term 1 FOLLOWER]: Refusing update from remote peer 7a011299d20d4ebfa87007fc0ef36f46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:37:23.472958 8950 raft_consensus.cc:1261] T ac33e7af9254487fb2ed25b611900057 P 27bd56c664f5442f801d24e4cbce13f8 [term 1 FOLLOWER]: Refusing update from remote peer 7a011299d20d4ebfa87007fc0ef36f46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:37:23.473330 9564 consensus_queue.cc:1035] T ac33e7af9254487fb2ed25b611900057 P 7a011299d20d4ebfa87007fc0ef36f46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a10d2415c1f14d0bbf3386dbcdc80698" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 33883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:37:23.473455 9567 consensus_queue.cc:1035] T ac33e7af9254487fb2ed25b611900057 P 7a011299d20d4ebfa87007fc0ef36f46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27bd56c664f5442f801d24e4cbce13f8" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 33889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 16:37:23.474336 9017 tablet.cc:2369] T ac33e7af9254487fb2ed25b611900057 P 27bd56c664f5442f801d24e4cbce13f8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:37:23.476029 9571 mvcc.cc:205] Tried to move back new op lower bound from 7317719627662376960 to 7317719627605032960. Current Snapshot: MvccSnapshot[applied={T|T < 7317719627662376960}]
I20260812 16:37:23.476460 9572 mvcc.cc:205] Tried to move back new op lower bound from 7317719627662376960 to 7317719627605032960. Current Snapshot: MvccSnapshot[applied={T|T < 7317719627662376960}]
I20260812 16:37:23.476622 9573 mvcc.cc:205] Tried to move back new op lower bound from 7317719627662376960 to 7317719627605032960. Current Snapshot: MvccSnapshot[applied={T|T < 7317719627662376960}]
I20260812 16:37:23.478639 8725 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:37:23.478739 8725 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:37:23.479660 8725 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1627ae4a6a764c6b908af4ed8d1e5a98: Sending DeleteTablet for 3 replicas of tablet ac33e7af9254487fb2ed25b611900057
I20260812 16:37:23.480021 9372 tablet_service.cc:1514] Processing DeleteTablet for tablet ac33e7af9254487fb2ed25b611900057 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:23 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:39114
I20260812 16:37:23.480022 8920 tablet_service.cc:1514] Processing DeleteTablet for tablet ac33e7af9254487fb2ed25b611900057 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:23 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:45928
I20260812 16:37:23.480288 9587 tablet_replica.cc:330] T ac33e7af9254487fb2ed25b611900057 P 27bd56c664f5442f801d24e4cbce13f8: stopping tablet replica
I20260812 16:37:23.480288 9132 tablet_service.cc:1514] Processing DeleteTablet for tablet ac33e7af9254487fb2ed25b611900057 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:23 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:53450
I20260812 16:37:23.480407 9587 raft_consensus.cc:2229] T ac33e7af9254487fb2ed25b611900057 P 27bd56c664f5442f801d24e4cbce13f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:37:23.480535 9587 raft_consensus.cc:2258] T ac33e7af9254487fb2ed25b611900057 P 27bd56c664f5442f801d24e4cbce13f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:23.480542 9589 tablet_replica.cc:330] T ac33e7af9254487fb2ed25b611900057 P 7a011299d20d4ebfa87007fc0ef36f46: stopping tablet replica
I20260812 16:37:23.480677 9589 raft_consensus.cc:2229] T ac33e7af9254487fb2ed25b611900057 P 7a011299d20d4ebfa87007fc0ef36f46 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:37:23.480822 9589 raft_consensus.cc:2258] T ac33e7af9254487fb2ed25b611900057 P 7a011299d20d4ebfa87007fc0ef36f46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:23.480903 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 8824
I20260812 16:37:23.481071 9587 ts_tablet_manager.cc:1905] T ac33e7af9254487fb2ed25b611900057 P 27bd56c664f5442f801d24e4cbce13f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:37:23.484810 9589 ts_tablet_manager.cc:1905] T ac33e7af9254487fb2ed25b611900057 P 7a011299d20d4ebfa87007fc0ef36f46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:37:23.486541 9589 ts_tablet_manager.cc:1918] T ac33e7af9254487fb2ed25b611900057 P 7a011299d20d4ebfa87007fc0ef36f46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:37:23.486661 9589 log.cc:1196] T ac33e7af9254487fb2ed25b611900057 P 7a011299d20d4ebfa87007fc0ef36f46: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-1/wal/wals/ac33e7af9254487fb2ed25b611900057
I20260812 16:37:23.486969 9589 ts_tablet_manager.cc:1939] T ac33e7af9254487fb2ed25b611900057 P 7a011299d20d4ebfa87007fc0ef36f46: Deleting consensus metadata
I20260812 16:37:23.487452 8703 catalog_manager.cc:4841] TS 7a011299d20d4ebfa87007fc0ef36f46 (127.0.253.194:38219): tablet ac33e7af9254487fb2ed25b611900057 (table test-table [id=67444bb2026248d982627d0dde10c0ec]) successfully deleted
I20260812 16:37:23.490029 9585 tablet_replica.cc:330] T ac33e7af9254487fb2ed25b611900057 P a10d2415c1f14d0bbf3386dbcdc80698: stopping tablet replica
I20260812 16:37:23.490186 9585 raft_consensus.cc:2229] T ac33e7af9254487fb2ed25b611900057 P a10d2415c1f14d0bbf3386dbcdc80698 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:37:23.490322 9585 raft_consensus.cc:2258] T ac33e7af9254487fb2ed25b611900057 P a10d2415c1f14d0bbf3386dbcdc80698 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:23.490840 9585 ts_tablet_manager.cc:1905] T ac33e7af9254487fb2ed25b611900057 P a10d2415c1f14d0bbf3386dbcdc80698: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:37:23.492493 9585 ts_tablet_manager.cc:1918] T ac33e7af9254487fb2ed25b611900057 P a10d2415c1f14d0bbf3386dbcdc80698: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:37:23.492640 9585 log.cc:1196] T ac33e7af9254487fb2ed25b611900057 P a10d2415c1f14d0bbf3386dbcdc80698: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552619305671-1015-0/minicluster-data/ts-2/wal/wals/ac33e7af9254487fb2ed25b611900057
I20260812 16:37:23.492986 9585 ts_tablet_manager.cc:1939] T ac33e7af9254487fb2ed25b611900057 P a10d2415c1f14d0bbf3386dbcdc80698: Deleting consensus metadata
I20260812 16:37:23.493520 8700 catalog_manager.cc:4841] TS a10d2415c1f14d0bbf3386dbcdc80698 (127.0.253.195:33883): tablet ac33e7af9254487fb2ed25b611900057 (table test-table [id=67444bb2026248d982627d0dde10c0ec]) successfully deleted
W20260812 16:37:23.495450 8698 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.253.193:33889 (error 108)
I20260812 16:37:23.496063 9550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:23.495882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:33889 (local address 127.0.0.1:45936)
0812 16:37:23.495947 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:23.495994 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.193:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 16:37:23.496135 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 9022
W20260812 16:37:23.499282 8698 catalog_manager.cc:4584] TS 27bd56c664f5442f801d24e4cbce13f8 (127.0.253.193:33889): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ac33e7af9254487fb2ed25b611900057: Network error: Client connection negotiation failed: client connection to 127.0.253.193:33889: connect: Connection refused (error 111)
I20260812 16:37:23.513258 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 9252
I20260812 16:37:23.529080 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 8671
2026-08-12T16:37:23Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2347 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-12T16:37:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:37:23Z Disabled control of system clock
I20260812 16:37:23.565102 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:46403
--webserver_interface=127.0.253.254
--webserver_port=0
--builtin_ntp_servers=127.0.253.212:46243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:46403
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 16:37:23.649288 9599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:23.649509 9599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:23.649533 9599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:23.650743 9599 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:37:23.650792 9599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:23.650807 9599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:23.650820 9599 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:37:23.650832 9599 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:37:23.652311 9599 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:46243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:46403
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:46403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:23.652596 9599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:23.652916 9599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:23.657972 9617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:23.658042 9616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:23.658042 9620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:23.658345 9599 server_base.cc:1034] running on GCE node
I20260812 16:37:23.658607 9599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:23.658896 9599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:23.660056 9599 hybrid_clock.cc:648] HybridClock initialized: now 1786552643660038 us; error 38 us; skew 500 ppm
I20260812 16:37:23.661293 9599 webserver.cc:460] Webserver started at http://127.0.253.254:42749/ using document root <none> and password file <none>
I20260812 16:37:23.661569 9599 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:23.661639 9599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:23.661811 9599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:23.662907 9599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "b74468b0e5cf467283979152003bb1c7"
format_stamp: "Formatted at 2026-08-12 16:37:23 on dist-test-slave-w4v5"
server_key: "cabaf39a3b212f563199a7d4b21b951b"
server_key_iv: "f8df4d6d9bac449ce76ec70ab2aec184"
server_key_version: "encryptionkey@0"
I20260812 16:37:23.663278 9599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "b74468b0e5cf467283979152003bb1c7"
format_stamp: "Formatted at 2026-08-12 16:37:23 on dist-test-slave-w4v5"
server_key: "cabaf39a3b212f563199a7d4b21b951b"
server_key_iv: "f8df4d6d9bac449ce76ec70ab2aec184"
server_key_version: "encryptionkey@0"
I20260812 16:37:23.664546 9599 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:37:23.665249 9632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:23.665436 9599 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 16:37:23.665537 9599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "b74468b0e5cf467283979152003bb1c7"
format_stamp: "Formatted at 2026-08-12 16:37:23 on dist-test-slave-w4v5"
server_key: "cabaf39a3b212f563199a7d4b21b951b"
server_key_iv: "f8df4d6d9bac449ce76ec70ab2aec184"
server_key_version: "encryptionkey@0"
I20260812 16:37:23.665608 9599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:23.669014 9599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:23.669278 9599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:23.669435 9599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:23.673249 9599 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:46403
I20260812 16:37:23.673285 9711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:46403 every 8 connection(s)
I20260812 16:37:23.673614 9599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:37:23.674304 9714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:23.677228 9714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b74468b0e5cf467283979152003bb1c7: Bootstrap starting.
I20260812 16:37:23.677845 9714 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b74468b0e5cf467283979152003bb1c7: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:23.678149 9714 log.cc:824] T 00000000000000000000000000000000 P b74468b0e5cf467283979152003bb1c7: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:23.678814 9714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b74468b0e5cf467283979152003bb1c7: No bootstrap required, opened a new log
I20260812 16:37:23.679713 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 9599
I20260812 16:37:23.679816 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:37:23.679991 1015 external_mini_cluster.cc:1428] Setting key e090d9b0110b057c1bb38dfe9831bf31
I20260812 16:37:23.681030 9714 raft_consensus.cc:348] T 00000000000000000000000000000000 P b74468b0e5cf467283979152003bb1c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b74468b0e5cf467283979152003bb1c7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 46403 } }
I20260812 16:37:23.681218 9714 raft_consensus.cc:374] T 00000000000000000000000000000000 P b74468b0e5cf467283979152003bb1c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:23.681272 9714 raft_consensus.cc:729] T 00000000000000000000000000000000 P b74468b0e5cf467283979152003bb1c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b74468b0e5cf467283979152003bb1c7, State: Initialized, Role: FOLLOWER
I20260812 16:37:23.681413 9714 consensus_queue.cc:260] T 00000000000000000000000000000000 P b74468b0e5cf467283979152003bb1c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b74468b0e5cf467283979152003bb1c7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 46403 } }
I20260812 16:37:23.681515 9714 raft_consensus.cc:388] T 00000000000000000000000000000000 P b74468b0e5cf467283979152003bb1c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:37:23.681553 9714 raft_consensus.cc:482] T 00000000000000000000000000000000 P b74468b0e5cf467283979152003bb1c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:37:23.681620 9714 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b74468b0e5cf467283979152003bb1c7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:23.682353 9714 raft_consensus.cc:504] T 00000000000000000000000000000000 P b74468b0e5cf467283979152003bb1c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b74468b0e5cf467283979152003bb1c7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 46403 } }
I20260812 16:37:23.682382 9719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:23.680501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34618 (local address 127.0.253.254:46403)
0812 16:37:23.680728 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:23.680735 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:23.680764 (+ 29us) server_negotiation.cc:407] Connection header received
0812 16:37:23.680988 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:23.680994 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:23.681035 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:23.681138 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:23.681930 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:23.681934 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:23.681934 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:37:23.681938 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:23.681975 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:37:23.681985 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:23.682045 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:23.682133 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 16:37:23.682195 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":94,"threads_started":1}
I20260812 16:37:23.682466 9714 leader_election.cc:302] T 00000000000000000000000000000000 P b74468b0e5cf467283979152003bb1c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b74468b0e5cf467283979152003bb1c7; no voters:
I20260812 16:37:23.682627 9714 leader_election.cc:288] T 00000000000000000000000000000000 P b74468b0e5cf467283979152003bb1c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:37:23.682881 9714 sys_catalog.cc:564] T 00000000000000000000000000000000 P b74468b0e5cf467283979152003bb1c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:37:23.682915 9721 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b74468b0e5cf467283979152003bb1c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:37:23.683039 9721 raft_consensus.cc:686] T 00000000000000000000000000000000 P b74468b0e5cf467283979152003bb1c7 [term 1 LEADER]: Becoming Leader. State: Replica: b74468b0e5cf467283979152003bb1c7, State: Running, Role: LEADER
I20260812 16:37:23.683138 9721 consensus_queue.cc:237] T 00000000000000000000000000000000 P b74468b0e5cf467283979152003bb1c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b74468b0e5cf467283979152003bb1c7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 46403 } }
I20260812 16:37:23.683498 9721 sys_catalog.cc:455] T 00000000000000000000000000000000 P b74468b0e5cf467283979152003bb1c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b74468b0e5cf467283979152003bb1c7. Latest consensus state: current_term: 1 leader_uuid: "b74468b0e5cf467283979152003bb1c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b74468b0e5cf467283979152003bb1c7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 46403 } } }
I20260812 16:37:23.683513 9724 sys_catalog.cc:455] T 00000000000000000000000000000000 P b74468b0e5cf467283979152003bb1c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b74468b0e5cf467283979152003bb1c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b74468b0e5cf467283979152003bb1c7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 46403 } } }
I20260812 16:37:23.683593 9721 sys_catalog.cc:458] T 00000000000000000000000000000000 P b74468b0e5cf467283979152003bb1c7 [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:23.683600 9724 sys_catalog.cc:458] T 00000000000000000000000000000000 P b74468b0e5cf467283979152003bb1c7 [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:23.683790 9734 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:37:23.684350 9734 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:37:23.685894 9734 catalog_manager.cc:1347] Generated new cluster ID: aabe58c666a043e9b210e75c1871930f
I20260812 16:37:23.685938 9734 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:37:23.701092 9734 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:37:23.701259 9734 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:37:23.709517 9734 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b74468b0e5cf467283979152003bb1c7: Generated new TSK 0
I20260812 16:37:23.709710 9734 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:37:23.716229 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.193:0
--local_ip_for_outbound_sockets=127.0.253.193
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:46403
--builtin_ntp_servers=127.0.253.212:46243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 16:37:23.802904 9749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:23.803176 9749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:23.803210 9749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:23.804365 9749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:23.804404 9749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:23.804428 9749 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.193
I20260812 16:37:23.805720 9749 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:46243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:46403
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9749
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.193
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:23.805905 9749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:23.806141 9749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 16:37:23.811439 9749 server_base.cc:1034] running on GCE node
W20260812 16:37:23.811408 9760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:23.811411 9759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:23.811403 9762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:23.811841 9749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:23.812088 9749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:23.813287 9749 hybrid_clock.cc:648] HybridClock initialized: now 1786552643813256 us; error 48 us; skew 500 ppm
I20260812 16:37:23.814742 9749 webserver.cc:460] Webserver started at http://127.0.253.193:36789/ using document root <none> and password file <none>
I20260812 16:37:23.814996 9749 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:23.815091 9749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:23.815245 9749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:23.816329 9749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-0/data/instance:
uuid: "1424b58856cd4370883faebec92cefac"
format_stamp: "Formatted at 2026-08-12 16:37:23 on dist-test-slave-w4v5"
server_key: "b47d7eb106f8d2121d8c195812459ffc"
server_key_iv: "f7ee1df31111ee50ce5869d004c2d069"
server_key_version: "encryptionkey@0"
I20260812 16:37:23.816711 9749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance:
uuid: "1424b58856cd4370883faebec92cefac"
format_stamp: "Formatted at 2026-08-12 16:37:23 on dist-test-slave-w4v5"
server_key: "b47d7eb106f8d2121d8c195812459ffc"
server_key_iv: "f7ee1df31111ee50ce5869d004c2d069"
server_key_version: "encryptionkey@0"
I20260812 16:37:23.818090 9749 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:37:23.818871 9783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:23.819062 9749 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:23.819214 9749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-0/wal
uuid: "1424b58856cd4370883faebec92cefac"
format_stamp: "Formatted at 2026-08-12 16:37:23 on dist-test-slave-w4v5"
server_key: "b47d7eb106f8d2121d8c195812459ffc"
server_key_iv: "f7ee1df31111ee50ce5869d004c2d069"
server_key_version: "encryptionkey@0"
I20260812 16:37:23.819289 9749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:23.822759 9749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:23.823031 9749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:23.823199 9749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:23.823414 9749 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:37:23.823732 9749 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:23.823783 9749 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:23.823823 9749 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:23.823850 9749 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:23.829854 9749 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.193:41183
I20260812 16:37:23.829938 9975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.193:41183 every 8 connection(s)
I20260812 16:37:23.830344 9749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
I20260812 16:37:23.832532 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 9749
I20260812 16:37:23.832644 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance
I20260812 16:37:23.832855 1015 external_mini_cluster.cc:1428] Setting key 9e57549b2cd2f83837a63372386fb5d6
I20260812 16:37:23.833799 9719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:23.831187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:45083 (local address 127.0.253.254:46403)
0812 16:37:23.831236 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:23.831238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:23.831511 (+ 273us) server_negotiation.cc:407] Connection header received
0812 16:37:23.832045 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:23.832046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:23.832055 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:23.832101 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:23.833514 (+ 1413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:23.833515 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:23.833516 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:37:23.833521 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:23.833538 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:37:23.833543 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:23.833593 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:23.833712 (+ 119us) server_negotiation.cc:299] Negotiation successful
0812 16:37:23.833730 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 16:37:23.834012 9979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:23.831191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:46403 (local address 127.0.253.193:45083)
0812 16:37:23.831391 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:23.831425 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:23.831936 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:23.832169 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:23.832174 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:23.832187 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:23.833365 (+ 1178us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:37:23.833365 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:23.833395 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:37:23.833424 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:23.833600 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:23.833605 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:23.833609 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:37:23.833723 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 16:37:23.833787 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":102,"threads_started":1}
I20260812 16:37:23.834275 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.194:0
--local_ip_for_outbound_sockets=127.0.253.194
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:46403
--builtin_ntp_servers=127.0.253.212:46243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 16:37:23.835490 9977 heartbeater.cc:347] Connected to a master server at 127.0.253.254:46403
I20260812 16:37:23.835592 9977 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:23.835764 9977 heartbeater.cc:511] Master 127.0.253.254:46403 requested a full tablet report, sending...
I20260812 16:37:23.836293 9659 ts_manager.cc:194] Registered new tserver with Master: 1424b58856cd4370883faebec92cefac (127.0.253.193:41183)
W20260812 16:37:23.920990 9980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:23.921232 9980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:23.921259 9980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:23.922544 9980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:23.922585 9980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:23.922611 9980 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.194
I20260812 16:37:23.924096 9980 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:46243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:46403
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9980
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.194
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:23.924310 9980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:23.924571 9980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:23.929800 10001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:23.929840 9998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:23.930003 9980 server_base.cc:1034] running on GCE node
W20260812 16:37:23.929823 9997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:23.930262 9980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:23.930526 9980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:23.931685 9980 hybrid_clock.cc:648] HybridClock initialized: now 1786552643931675 us; error 23 us; skew 500 ppm
I20260812 16:37:23.933061 9980 webserver.cc:460] Webserver started at http://127.0.253.194:43199/ using document root <none> and password file <none>
I20260812 16:37:23.933339 9980 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:23.933411 9980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:23.933558 9980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:23.934655 9980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-1/data/instance:
uuid: "a470c7f622a4498a83f023c377b85df4"
format_stamp: "Formatted at 2026-08-12 16:37:23 on dist-test-slave-w4v5"
server_key: "57d326f4397cceccffccd246eea15a4e"
server_key_iv: "11665c1d5528f9b927094ce72ee55c4c"
server_key_version: "encryptionkey@0"
I20260812 16:37:23.935030 9980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance:
uuid: "a470c7f622a4498a83f023c377b85df4"
format_stamp: "Formatted at 2026-08-12 16:37:23 on dist-test-slave-w4v5"
server_key: "57d326f4397cceccffccd246eea15a4e"
server_key_iv: "11665c1d5528f9b927094ce72ee55c4c"
server_key_version: "encryptionkey@0"
I20260812 16:37:23.936421 9980 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:37:23.937235 10012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:23.937431 9980 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:37:23.937536 9980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-1/wal
uuid: "a470c7f622a4498a83f023c377b85df4"
format_stamp: "Formatted at 2026-08-12 16:37:23 on dist-test-slave-w4v5"
server_key: "57d326f4397cceccffccd246eea15a4e"
server_key_iv: "11665c1d5528f9b927094ce72ee55c4c"
server_key_version: "encryptionkey@0"
I20260812 16:37:23.937608 9980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:23.941074 9980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:23.941341 9980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:23.941478 9980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:23.941677 9980 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:37:23.941988 9980 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:23.942039 9980 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:23.942080 9980 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:23.942106 9980 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:23.947875 9980 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.194:45711
I20260812 16:37:23.947944 10215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.194:45711 every 8 connection(s)
I20260812 16:37:23.948271 9980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
I20260812 16:37:23.950857 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 9980
I20260812 16:37:23.950968 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance
I20260812 16:37:23.951201 1015 external_mini_cluster.cc:1428] Setting key 7df90cde1356e4e6d5e6f86cc48b7064
I20260812 16:37:23.951468 9719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:23.949018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:49505 (local address 127.0.253.254:46403)
0812 16:37:23.949066 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:23.949068 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:23.949272 (+ 204us) server_negotiation.cc:407] Connection header received
0812 16:37:23.949763 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:23.949764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:23.949770 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:23.949821 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:23.951152 (+ 1331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:23.951153 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:23.951154 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:37:23.951159 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:23.951175 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:37:23.951179 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:23.951233 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:23.951365 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 16:37:23.951380 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 16:37:23.951661 10221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:23.948993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:46403 (local address 127.0.253.194:49505)
0812 16:37:23.949166 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:23.949198 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:23.949669 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:23.949874 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:23.949877 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:23.949885 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:23.950909 (+ 1024us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:37:23.950910 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:23.950955 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:37:23.950993 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:23.951244 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:23.951251 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:23.951255 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:37:23.951372 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 16:37:23.951442 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":90,"threads_started":1}
I20260812 16:37:23.952095 10216 heartbeater.cc:347] Connected to a master server at 127.0.253.254:46403
I20260812 16:37:23.952199 10216 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:23.952333 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.195:0
--local_ip_for_outbound_sockets=127.0.253.195
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:46403
--builtin_ntp_servers=127.0.253.212:46243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 16:37:23.952350 10216 heartbeater.cc:511] Master 127.0.253.254:46403 requested a full tablet report, sending...
I20260812 16:37:23.952724 9659 ts_manager.cc:194] Registered new tserver with Master: a470c7f622a4498a83f023c377b85df4 (127.0.253.194:45711)
W20260812 16:37:24.037808 10224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:24.038034 10224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:24.038059 10224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:24.039328 10224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:24.039376 10224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:24.039403 10224 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.195
I20260812 16:37:24.040910 10224 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:46243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:46403
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.195
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:24.041195 10224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:24.041551 10224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:24.046945 10239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:24.046988 10236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:24.047158 10224 server_base.cc:1034] running on GCE node
W20260812 16:37:24.046952 10234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:24.047447 10224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:24.047737 10224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:24.048907 10224 hybrid_clock.cc:648] HybridClock initialized: now 1786552644048894 us; error 28 us; skew 500 ppm
I20260812 16:37:24.050220 10224 webserver.cc:460] Webserver started at http://127.0.253.195:44719/ using document root <none> and password file <none>
I20260812 16:37:24.050439 10224 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:24.050498 10224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:24.050598 10224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:24.051744 10224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-2/data/instance:
uuid: "62a1a7bcde864bb6a8a45b0050ab9a26"
format_stamp: "Formatted at 2026-08-12 16:37:24 on dist-test-slave-w4v5"
server_key: "f99d873e36dbed3a45ef4fa23bd44fa0"
server_key_iv: "9653f72cb53e098c6e5fba56e61e6638"
server_key_version: "encryptionkey@0"
I20260812 16:37:24.052065 10224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance:
uuid: "62a1a7bcde864bb6a8a45b0050ab9a26"
format_stamp: "Formatted at 2026-08-12 16:37:24 on dist-test-slave-w4v5"
server_key: "f99d873e36dbed3a45ef4fa23bd44fa0"
server_key_iv: "9653f72cb53e098c6e5fba56e61e6638"
server_key_version: "encryptionkey@0"
I20260812 16:37:24.053381 10224 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:24.054111 10248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:24.054299 10224 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:37:24.054389 10224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-2/wal
uuid: "62a1a7bcde864bb6a8a45b0050ab9a26"
format_stamp: "Formatted at 2026-08-12 16:37:24 on dist-test-slave-w4v5"
server_key: "f99d873e36dbed3a45ef4fa23bd44fa0"
server_key_iv: "9653f72cb53e098c6e5fba56e61e6638"
server_key_version: "encryptionkey@0"
I20260812 16:37:24.054493 10224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:24.057915 10224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:24.058162 10224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:24.058260 10224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:24.058418 10224 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:37:24.058668 10224 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:24.058698 10224 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:24.058717 10224 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:24.058730 10224 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:24.064471 10224 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.195:41683
I20260812 16:37:24.064539 10440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.195:41683 every 8 connection(s)
I20260812 16:37:24.064781 10224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
I20260812 16:37:24.067968 9719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:24.065513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:56353 (local address 127.0.253.254:46403)
0812 16:37:24.065577 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:24.065579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:24.065851 (+ 272us) server_negotiation.cc:407] Connection header received
0812 16:37:24.066364 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:24.066366 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:24.066374 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:24.066425 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:24.067715 (+ 1290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:24.067717 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:24.067717 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:37:24.067722 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:24.067739 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:37:24.067743 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:24.067797 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:24.067882 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 16:37:24.067899 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 16:37:24.068185 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 10224
I20260812 16:37:24.068220 10449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:24.065513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:46403 (local address 127.0.253.195:56353)
0812 16:37:24.065699 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:24.065761 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:24.066248 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:24.066495 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:24.066503 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:24.066512 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:24.067533 (+ 1021us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:37:24.067533 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:24.067564 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:37:24.067592 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:24.067792 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:24.067796 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:24.067800 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:37:24.067892 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 16:37:24.067954 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":69,"threads_started":1}
I20260812 16:37:24.068290 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance
I20260812 16:37:24.068486 1015 external_mini_cluster.cc:1428] Setting key d3b7ad141cf1c7106fc5658811fe658a
I20260812 16:37:24.068686 10442 heartbeater.cc:347] Connected to a master server at 127.0.253.254:46403
I20260812 16:37:24.068804 10442 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:24.068959 10442 heartbeater.cc:511] Master 127.0.253.254:46403 requested a full tablet report, sending...
I20260812 16:37:24.069365 9659 ts_manager.cc:194] Registered new tserver with Master: 62a1a7bcde864bb6a8a45b0050ab9a26 (127.0.253.195:41683)
I20260812 16:37:24.080410 1015 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:37:24.083036 9719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:24.081838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34626 (local address 127.0.253.254:46403)
0812 16:37:24.081892 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:24.081898 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:24.082034 (+ 136us) server_negotiation.cc:407] Connection header received
0812 16:37:24.082063 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:24.082064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:24.082069 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:24.082122 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:24.082794 (+ 672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:24.082795 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:24.082795 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:37:24.082799 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:24.082815 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:37:24.082818 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:24.082894 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:24.082967 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 16:37:24.082986 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 16:37:24.084616 9659 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34626:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:37:24.086318 9659 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:37:24.090247 10459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:24.088614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:45711 (local address 127.0.0.1:54468)
0812 16:37:24.088853 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:24.088894 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:24.088956 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:24.089361 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:24.089367 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:24.089373 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:24.089809 (+ 436us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:37:24.089810 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:24.089826 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:37:24.089853 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:24.090111 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:24.090117 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:24.090120 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:37:24.090168 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 16:37:24.090181 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":76,"threads_started":1}
I20260812 16:37:24.090672 10457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:24.088673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.195:41683 (local address 127.0.0.1:52264)
0812 16:37:24.088933 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:24.088946 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:24.088992 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:24.089930 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:24.089931 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:24.089933 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:24.090280 (+ 347us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:37:24.090281 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:24.090289 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:37:24.090303 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:24.090539 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:24.090540 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:24.090541 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 16:37:24.090600 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 16:37:24.090612 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":101,"threads_started":1}
I20260812 16:37:24.090737 10461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:24.088774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52264 (local address 127.0.253.195:41683)
0812 16:37:24.089779 (+ 1005us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:24.089783 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:24.089793 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:37:24.089817 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:24.089820 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:24.089828 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:24.089911 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:24.090395 (+ 484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:24.090405 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:24.090406 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:37:24.090410 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:24.090444 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:37:24.090468 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:24.090541 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:24.090609 (+ 68us) server_negotiation.cc:299] Negotiation successful
0812 16:37:24.090626 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":966,"thread_start_us":78,"threads_started":1}
I20260812 16:37:24.090696 10460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:24.088738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54468 (local address 127.0.253.194:45711)
0812 16:37:24.089211 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:24.089216 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:24.089223 (+ 7us) server_negotiation.cc:407] Connection header received
0812 16:37:24.089264 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:24.089266 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:24.089276 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:24.089347 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:24.089943 (+ 596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:24.089954 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:24.089954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:37:24.089959 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:24.089997 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:37:24.090028 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:24.090555 (+ 527us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:24.090582 (+ 27us) server_negotiation.cc:299] Negotiation successful
0812 16:37:24.090601 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":98,"threads_started":1}
I20260812 16:37:24.091106 10463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:24.088837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:41183 (local address 127.0.0.1:39234)
0812 16:37:24.089051 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:24.089060 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:24.089093 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:24.089783 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:24.089784 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:24.089787 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:24.090529 (+ 742us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:37:24.090529 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:24.090535 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:37:24.090546 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:24.090940 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:24.090941 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:24.090942 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 16:37:24.091011 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 16:37:24.091025 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":65,"threads_started":1}
I20260812 16:37:24.091298 10467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:24.089056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39234 (local address 127.0.253.193:41183)
0812 16:37:24.089249 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:24.089253 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:24.089264 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:37:24.089291 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:24.089293 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:24.089313 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:24.089392 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:24.090718 (+ 1326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:24.090722 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:24.090722 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:37:24.090725 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:24.090841 (+ 116us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:37:24.090863 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:24.090945 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:24.091201 (+ 256us) server_negotiation.cc:299] Negotiation successful
0812 16:37:24.091220 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":104,"threads_started":1}
I20260812 16:37:24.091604 10326 tablet_service.cc:1467] Processing CreateTablet for tablet d851db6d5c6f4581ae640c4df172e746 (DEFAULT_TABLE table=test-table [id=cdf6c678138b41a68f3fe1ed9c188a68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:24.091768 10099 tablet_service.cc:1467] Processing CreateTablet for tablet d851db6d5c6f4581ae640c4df172e746 (DEFAULT_TABLE table=test-table [id=cdf6c678138b41a68f3fe1ed9c188a68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:24.091917 10326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d851db6d5c6f4581ae640c4df172e746. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:24.091953 9861 tablet_service.cc:1467] Processing CreateTablet for tablet d851db6d5c6f4581ae640c4df172e746 (DEFAULT_TABLE table=test-table [id=cdf6c678138b41a68f3fe1ed9c188a68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:24.092127 10099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d851db6d5c6f4581ae640c4df172e746. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:24.092278 9861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d851db6d5c6f4581ae640c4df172e746. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:24.094347 10469 tablet_bootstrap.cc:492] T d851db6d5c6f4581ae640c4df172e746 P 62a1a7bcde864bb6a8a45b0050ab9a26: Bootstrap starting.
I20260812 16:37:24.094959 10469 tablet_bootstrap.cc:654] T d851db6d5c6f4581ae640c4df172e746 P 62a1a7bcde864bb6a8a45b0050ab9a26: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:24.095315 10469 log.cc:824] T d851db6d5c6f4581ae640c4df172e746 P 62a1a7bcde864bb6a8a45b0050ab9a26: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:24.095674 10471 tablet_bootstrap.cc:492] T d851db6d5c6f4581ae640c4df172e746 P 1424b58856cd4370883faebec92cefac: Bootstrap starting.
I20260812 16:37:24.096077 10469 tablet_bootstrap.cc:492] T d851db6d5c6f4581ae640c4df172e746 P 62a1a7bcde864bb6a8a45b0050ab9a26: No bootstrap required, opened a new log
I20260812 16:37:24.096133 10469 ts_tablet_manager.cc:1397] T d851db6d5c6f4581ae640c4df172e746 P 62a1a7bcde864bb6a8a45b0050ab9a26: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 16:37:24.096318 10471 tablet_bootstrap.cc:654] T d851db6d5c6f4581ae640c4df172e746 P 1424b58856cd4370883faebec92cefac: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:24.096614 10471 log.cc:824] T d851db6d5c6f4581ae640c4df172e746 P 1424b58856cd4370883faebec92cefac: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:24.096992 10470 tablet_bootstrap.cc:492] T d851db6d5c6f4581ae640c4df172e746 P a470c7f622a4498a83f023c377b85df4: Bootstrap starting.
I20260812 16:37:24.097487 10471 tablet_bootstrap.cc:492] T d851db6d5c6f4581ae640c4df172e746 P 1424b58856cd4370883faebec92cefac: No bootstrap required, opened a new log
I20260812 16:37:24.097528 10470 tablet_bootstrap.cc:654] T d851db6d5c6f4581ae640c4df172e746 P a470c7f622a4498a83f023c377b85df4: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:24.097613 10471 ts_tablet_manager.cc:1397] T d851db6d5c6f4581ae640c4df172e746 P 1424b58856cd4370883faebec92cefac: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:37:24.097855 10470 log.cc:824] T d851db6d5c6f4581ae640c4df172e746 P a470c7f622a4498a83f023c377b85df4: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:24.098513 10470 tablet_bootstrap.cc:492] T d851db6d5c6f4581ae640c4df172e746 P a470c7f622a4498a83f023c377b85df4: No bootstrap required, opened a new log
I20260812 16:37:24.098593 10470 ts_tablet_manager.cc:1397] T d851db6d5c6f4581ae640c4df172e746 P a470c7f622a4498a83f023c377b85df4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:37:24.098655 10469 raft_consensus.cc:348] T d851db6d5c6f4581ae640c4df172e746 P 62a1a7bcde864bb6a8a45b0050ab9a26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a470c7f622a4498a83f023c377b85df4" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 45711 } } peers { permanent_uuid: "1424b58856cd4370883faebec92cefac" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 41183 } } peers { permanent_uuid: "62a1a7bcde864bb6a8a45b0050ab9a26" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 41683 } }
I20260812 16:37:24.098832 10469 raft_consensus.cc:374] T d851db6d5c6f4581ae640c4df172e746 P 62a1a7bcde864bb6a8a45b0050ab9a26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:24.098875 10469 raft_consensus.cc:729] T d851db6d5c6f4581ae640c4df172e746 P 62a1a7bcde864bb6a8a45b0050ab9a26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62a1a7bcde864bb6a8a45b0050ab9a26, State: Initialized, Role: FOLLOWER
I20260812 16:37:24.099009 10469 consensus_queue.cc:260] T d851db6d5c6f4581ae640c4df172e746 P 62a1a7bcde864bb6a8a45b0050ab9a26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a470c7f622a4498a83f023c377b85df4" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 45711 } } peers { permanent_uuid: "1424b58856cd4370883faebec92cefac" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 41183 } } peers { permanent_uuid: "62a1a7bcde864bb6a8a45b0050ab9a26" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 41683 } }
I20260812 16:37:24.099306 10469 ts_tablet_manager.cc:1428] T d851db6d5c6f4581ae640c4df172e746 P 62a1a7bcde864bb6a8a45b0050ab9a26: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260812 16:37:24.099452 10442 heartbeater.cc:503] Master 127.0.253.254:46403 was elected leader, sending a full tablet report...
I20260812 16:37:24.100353 10471 raft_consensus.cc:348] T d851db6d5c6f4581ae640c4df172e746 P 1424b58856cd4370883faebec92cefac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a470c7f622a4498a83f023c377b85df4" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 45711 } } peers { permanent_uuid: "1424b58856cd4370883faebec92cefac" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 41183 } } peers { permanent_uuid: "62a1a7bcde864bb6a8a45b0050ab9a26" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 41683 } }
I20260812 16:37:24.100467 10471 raft_consensus.cc:374] T d851db6d5c6f4581ae640c4df172e746 P 1424b58856cd4370883faebec92cefac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:24.100488 10471 raft_consensus.cc:729] T d851db6d5c6f4581ae640c4df172e746 P 1424b58856cd4370883faebec92cefac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1424b58856cd4370883faebec92cefac, State: Initialized, Role: FOLLOWER
I20260812 16:37:24.100589 10471 consensus_queue.cc:260] T d851db6d5c6f4581ae640c4df172e746 P 1424b58856cd4370883faebec92cefac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a470c7f622a4498a83f023c377b85df4" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 45711 } } peers { permanent_uuid: "1424b58856cd4370883faebec92cefac" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 41183 } } peers { permanent_uuid: "62a1a7bcde864bb6a8a45b0050ab9a26" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 41683 } }
I20260812 16:37:24.100793 10471 ts_tablet_manager.cc:1428] T d851db6d5c6f4581ae640c4df172e746 P 1424b58856cd4370883faebec92cefac: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 16:37:24.100901 9977 heartbeater.cc:503] Master 127.0.253.254:46403 was elected leader, sending a full tablet report...
I20260812 16:37:24.101191 10470 raft_consensus.cc:348] T d851db6d5c6f4581ae640c4df172e746 P a470c7f622a4498a83f023c377b85df4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a470c7f622a4498a83f023c377b85df4" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 45711 } } peers { permanent_uuid: "1424b58856cd4370883faebec92cefac" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 41183 } } peers { permanent_uuid: "62a1a7bcde864bb6a8a45b0050ab9a26" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 41683 } }
I20260812 16:37:24.101379 10470 raft_consensus.cc:374] T d851db6d5c6f4581ae640c4df172e746 P a470c7f622a4498a83f023c377b85df4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:24.101429 10470 raft_consensus.cc:729] T d851db6d5c6f4581ae640c4df172e746 P a470c7f622a4498a83f023c377b85df4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a470c7f622a4498a83f023c377b85df4, State: Initialized, Role: FOLLOWER
I20260812 16:37:24.101542 10470 consensus_queue.cc:260] T d851db6d5c6f4581ae640c4df172e746 P a470c7f622a4498a83f023c377b85df4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a470c7f622a4498a83f023c377b85df4" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 45711 } } peers { permanent_uuid: "1424b58856cd4370883faebec92cefac" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 41183 } } peers { permanent_uuid: "62a1a7bcde864bb6a8a45b0050ab9a26" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 41683 } }
I20260812 16:37:24.101836 10470 ts_tablet_manager.cc:1428] T d851db6d5c6f4581ae640c4df172e746 P a470c7f622a4498a83f023c377b85df4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:37:24.101955 10216 heartbeater.cc:503] Master 127.0.253.254:46403 was elected leader, sending a full tablet report...
W20260812 16:37:24.198832 10218 tablet.cc:2369] T d851db6d5c6f4581ae640c4df172e746 P a470c7f622a4498a83f023c377b85df4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 16:37:24.315264 10446 tablet.cc:2369] T d851db6d5c6f4581ae640c4df172e746 P 62a1a7bcde864bb6a8a45b0050ab9a26: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 16:37:24.331161 9978 tablet.cc:2369] T d851db6d5c6f4581ae640c4df172e746 P 1424b58856cd4370883faebec92cefac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:37:24.347458 10483 raft_consensus.cc:482] T d851db6d5c6f4581ae640c4df172e746 P a470c7f622a4498a83f023c377b85df4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:37:24.347587 10483 raft_consensus.cc:504] T d851db6d5c6f4581ae640c4df172e746 P a470c7f622a4498a83f023c377b85df4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a470c7f622a4498a83f023c377b85df4" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 45711 } } peers { permanent_uuid: "1424b58856cd4370883faebec92cefac" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 41183 } } peers { permanent_uuid: "62a1a7bcde864bb6a8a45b0050ab9a26" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 41683 } }
I20260812 16:37:24.348035 10483 leader_election.cc:288] T d851db6d5c6f4581ae640c4df172e746 P a470c7f622a4498a83f023c377b85df4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1424b58856cd4370883faebec92cefac (127.0.253.193:41183), 62a1a7bcde864bb6a8a45b0050ab9a26 (127.0.253.195:41683)
I20260812 16:37:24.349514 10221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:24.348225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:41183 (local address 127.0.253.194:41037)
0812 16:37:24.348276 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:24.348290 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:24.348338 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:24.348570 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:24.348570 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:24.348573 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:24.349099 (+ 526us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:37:24.349099 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:24.349109 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:37:24.349123 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:24.349334 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:24.349335 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:24.349336 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 16:37:24.349403 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 16:37:24.349420 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260812 16:37:24.349522 10467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:24.348376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:41037 (local address 127.0.253.193:41183)
0812 16:37:24.348420 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:24.348422 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:24.348431 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:37:24.348463 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:24.348463 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:24.348469 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:24.348550 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:24.349235 (+ 685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:24.349237 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:24.349237 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:37:24.349244 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:24.349262 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:37:24.349267 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:24.349327 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:24.349403 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 16:37:24.349420 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 16:37:24.349654 10461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:24.348308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:42583 (local address 127.0.253.195:41683)
0812 16:37:24.348362 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:24.348365 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:24.348527 (+ 162us) server_negotiation.cc:407] Connection header received
0812 16:37:24.348562 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:24.348563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:24.348568 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:24.348631 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:24.349412 (+ 781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:24.349413 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:24.349414 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:37:24.349418 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:24.349431 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:37:24.349435 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:24.349467 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:24.349574 (+ 107us) server_negotiation.cc:299] Negotiation successful
0812 16:37:24.349587 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 16:37:24.349856 10484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:24.348241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.195:41683 (local address 127.0.253.194:42583)
0812 16:37:24.348456 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:24.348464 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:24.348504 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:24.348639 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:24.348640 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:24.348661 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:24.349281 (+ 620us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:37:24.349282 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:24.349287 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:37:24.349298 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:24.349510 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:24.349511 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:24.349511 (+ 0us) client_negotiation.cc:769] Sending connection context
0812 16:37:24.349739 (+ 228us) client_negotiation.cc:240] Negotiation successful
0812 16:37:24.349755 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":58,"thread_start_us":109,"threads_started":1}
I20260812 16:37:24.350072 9899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d851db6d5c6f4581ae640c4df172e746" candidate_uuid: "a470c7f622a4498a83f023c377b85df4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1424b58856cd4370883faebec92cefac" is_pre_election: true
I20260812 16:37:24.350232 9899 raft_consensus.cc:2454] T d851db6d5c6f4581ae640c4df172e746 P 1424b58856cd4370883faebec92cefac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a470c7f622a4498a83f023c377b85df4 in term 0.
I20260812 16:37:24.350200 10362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d851db6d5c6f4581ae640c4df172e746" candidate_uuid: "a470c7f622a4498a83f023c377b85df4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62a1a7bcde864bb6a8a45b0050ab9a26" is_pre_election: true
I20260812 16:37:24.350349 10362 raft_consensus.cc:2454] T d851db6d5c6f4581ae640c4df172e746 P 62a1a7bcde864bb6a8a45b0050ab9a26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a470c7f622a4498a83f023c377b85df4 in term 0.
I20260812 16:37:24.350473 10018 leader_election.cc:302] T d851db6d5c6f4581ae640c4df172e746 P a470c7f622a4498a83f023c377b85df4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1424b58856cd4370883faebec92cefac, a470c7f622a4498a83f023c377b85df4; no voters:
I20260812 16:37:24.350615 10483 raft_consensus.cc:2781] T d851db6d5c6f4581ae640c4df172e746 P a470c7f622a4498a83f023c377b85df4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:37:24.350672 10483 raft_consensus.cc:482] T d851db6d5c6f4581ae640c4df172e746 P a470c7f622a4498a83f023c377b85df4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:37:24.350703 10483 raft_consensus.cc:3037] T d851db6d5c6f4581ae640c4df172e746 P a470c7f622a4498a83f023c377b85df4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:24.351507 10483 raft_consensus.cc:504] T d851db6d5c6f4581ae640c4df172e746 P a470c7f622a4498a83f023c377b85df4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a470c7f622a4498a83f023c377b85df4" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 45711 } } peers { permanent_uuid: "1424b58856cd4370883faebec92cefac" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 41183 } } peers { permanent_uuid: "62a1a7bcde864bb6a8a45b0050ab9a26" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 41683 } }
I20260812 16:37:24.351651 10483 leader_election.cc:288] T d851db6d5c6f4581ae640c4df172e746 P a470c7f622a4498a83f023c377b85df4 [CANDIDATE]: Term 1 election: Requested vote from peers 1424b58856cd4370883faebec92cefac (127.0.253.193:41183), 62a1a7bcde864bb6a8a45b0050ab9a26 (127.0.253.195:41683)
I20260812 16:37:24.351852 10362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d851db6d5c6f4581ae640c4df172e746" candidate_uuid: "a470c7f622a4498a83f023c377b85df4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62a1a7bcde864bb6a8a45b0050ab9a26"
I20260812 16:37:24.351837 9899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d851db6d5c6f4581ae640c4df172e746" candidate_uuid: "a470c7f622a4498a83f023c377b85df4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1424b58856cd4370883faebec92cefac"
I20260812 16:37:24.351922 10362 raft_consensus.cc:3037] T d851db6d5c6f4581ae640c4df172e746 P 62a1a7bcde864bb6a8a45b0050ab9a26 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:24.351922 9899 raft_consensus.cc:3037] T d851db6d5c6f4581ae640c4df172e746 P 1424b58856cd4370883faebec92cefac [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:24.352842 9899 raft_consensus.cc:2454] T d851db6d5c6f4581ae640c4df172e746 P 1424b58856cd4370883faebec92cefac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a470c7f622a4498a83f023c377b85df4 in term 1.
I20260812 16:37:24.352842 10362 raft_consensus.cc:2454] T d851db6d5c6f4581ae640c4df172e746 P 62a1a7bcde864bb6a8a45b0050ab9a26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a470c7f622a4498a83f023c377b85df4 in term 1.
I20260812 16:37:24.353011 10018 leader_election.cc:302] T d851db6d5c6f4581ae640c4df172e746 P a470c7f622a4498a83f023c377b85df4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1424b58856cd4370883faebec92cefac, a470c7f622a4498a83f023c377b85df4; no voters:
I20260812 16:37:24.353118 10483 raft_consensus.cc:2781] T d851db6d5c6f4581ae640c4df172e746 P a470c7f622a4498a83f023c377b85df4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:37:24.353251 10483 raft_consensus.cc:686] T d851db6d5c6f4581ae640c4df172e746 P a470c7f622a4498a83f023c377b85df4 [term 1 LEADER]: Becoming Leader. State: Replica: a470c7f622a4498a83f023c377b85df4, State: Running, Role: LEADER
I20260812 16:37:24.353353 10483 consensus_queue.cc:237] T d851db6d5c6f4581ae640c4df172e746 P a470c7f622a4498a83f023c377b85df4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a470c7f622a4498a83f023c377b85df4" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 45711 } } peers { permanent_uuid: "1424b58856cd4370883faebec92cefac" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 41183 } } peers { permanent_uuid: "62a1a7bcde864bb6a8a45b0050ab9a26" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 41683 } }
I20260812 16:37:24.354251 9658 catalog_manager.cc:5495] T d851db6d5c6f4581ae640c4df172e746 P a470c7f622a4498a83f023c377b85df4 reported cstate change: term changed from 0 to 1, leader changed from <none> to a470c7f622a4498a83f023c377b85df4 (127.0.253.194). New cstate: current_term: 1 leader_uuid: "a470c7f622a4498a83f023c377b85df4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a470c7f622a4498a83f023c377b85df4" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 45711 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1424b58856cd4370883faebec92cefac" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 41183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62a1a7bcde864bb6a8a45b0050ab9a26" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 41683 } health_report { overall_health: UNKNOWN } } }
I20260812 16:37:24.377642 10460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:24.376736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54476 (local address 127.0.253.194:45711)
0812 16:37:24.376790 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:24.376792 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:24.376800 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:37:24.376836 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:24.376837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:24.376850 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:24.376902 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:24.377459 (+ 557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:24.377460 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:24.377460 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:37:24.377463 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:24.377476 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:37:24.377480 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:24.377524 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:24.377572 (+ 48us) server_negotiation.cc:299] Negotiation successful
0812 16:37:24.377583 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 16:37:24.379591 10362 raft_consensus.cc:1261] T d851db6d5c6f4581ae640c4df172e746 P 62a1a7bcde864bb6a8a45b0050ab9a26 [term 1 FOLLOWER]: Refusing update from remote peer a470c7f622a4498a83f023c377b85df4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:37:24.379617 9899 raft_consensus.cc:1261] T d851db6d5c6f4581ae640c4df172e746 P 1424b58856cd4370883faebec92cefac [term 1 FOLLOWER]: Refusing update from remote peer a470c7f622a4498a83f023c377b85df4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:37:24.379976 10485 consensus_queue.cc:1035] T d851db6d5c6f4581ae640c4df172e746 P a470c7f622a4498a83f023c377b85df4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62a1a7bcde864bb6a8a45b0050ab9a26" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 41683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:37:24.380128 10483 consensus_queue.cc:1035] T d851db6d5c6f4581ae640c4df172e746 P a470c7f622a4498a83f023c377b85df4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1424b58856cd4370883faebec92cefac" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 41183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:37:24.382647 10490 mvcc.cc:205] Tried to move back new op lower bound from 7317719631375835136 to 7317719631271829504. Current Snapshot: MvccSnapshot[applied={T|T < 7317719631375835136}]
I20260812 16:37:24.383387 10493 mvcc.cc:205] Tried to move back new op lower bound from 7317719631375835136 to 7317719631271829504. Current Snapshot: MvccSnapshot[applied={T|T < 7317719631375835136}]
I20260812 16:37:24.385429 9657 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:37:24.385511 9657 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:37:24.386276 9657 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b74468b0e5cf467283979152003bb1c7: Sending DeleteTablet for 3 replicas of tablet d851db6d5c6f4581ae640c4df172e746
I20260812 16:37:24.386615 10326 tablet_service.cc:1514] Processing DeleteTablet for tablet d851db6d5c6f4581ae640c4df172e746 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:24 UTC) from {username='slave'} at 127.0.0.1:52264
I20260812 16:37:24.386662 9861 tablet_service.cc:1514] Processing DeleteTablet for tablet d851db6d5c6f4581ae640c4df172e746 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:24 UTC) from {username='slave'} at 127.0.0.1:39234
I20260812 16:37:24.386734 10099 tablet_service.cc:1514] Processing DeleteTablet for tablet d851db6d5c6f4581ae640c4df172e746 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:24 UTC) from {username='slave'} at 127.0.0.1:54468
I20260812 16:37:24.386896 10518 tablet_replica.cc:330] T d851db6d5c6f4581ae640c4df172e746 P 1424b58856cd4370883faebec92cefac: stopping tablet replica
I20260812 16:37:24.386998 10521 tablet_replica.cc:330] T d851db6d5c6f4581ae640c4df172e746 P a470c7f622a4498a83f023c377b85df4: stopping tablet replica
I20260812 16:37:24.387003 10518 raft_consensus.cc:2229] T d851db6d5c6f4581ae640c4df172e746 P 1424b58856cd4370883faebec92cefac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:37:24.387110 10521 raft_consensus.cc:2229] T d851db6d5c6f4581ae640c4df172e746 P a470c7f622a4498a83f023c377b85df4 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:37:24.387148 10518 raft_consensus.cc:2258] T d851db6d5c6f4581ae640c4df172e746 P 1424b58856cd4370883faebec92cefac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:24.387187 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 9749
I20260812 16:37:24.387231 10521 raft_consensus.cc:2258] T d851db6d5c6f4581ae640c4df172e746 P a470c7f622a4498a83f023c377b85df4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:24.387674 10521 ts_tablet_manager.cc:1905] T d851db6d5c6f4581ae640c4df172e746 P a470c7f622a4498a83f023c377b85df4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:37:24.390617 10521 ts_tablet_manager.cc:1918] T d851db6d5c6f4581ae640c4df172e746 P a470c7f622a4498a83f023c377b85df4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:37:24.390671 10521 log.cc:1196] T d851db6d5c6f4581ae640c4df172e746 P a470c7f622a4498a83f023c377b85df4: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-1/wal/wals/d851db6d5c6f4581ae640c4df172e746
I20260812 16:37:24.390893 10521 ts_tablet_manager.cc:1939] T d851db6d5c6f4581ae640c4df172e746 P a470c7f622a4498a83f023c377b85df4: Deleting consensus metadata
I20260812 16:37:24.406256 9639 catalog_manager.cc:4841] TS a470c7f622a4498a83f023c377b85df4 (127.0.253.194:45711): tablet d851db6d5c6f4581ae640c4df172e746 (table test-table [id=cdf6c678138b41a68f3fe1ed9c188a68]) successfully deleted
I20260812 16:37:24.387138 10516 tablet_replica.cc:330] T d851db6d5c6f4581ae640c4df172e746 P 62a1a7bcde864bb6a8a45b0050ab9a26: stopping tablet replica
I20260812 16:37:24.408131 10516 raft_consensus.cc:2229] T d851db6d5c6f4581ae640c4df172e746 P 62a1a7bcde864bb6a8a45b0050ab9a26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:37:24.408296 10516 raft_consensus.cc:2258] T d851db6d5c6f4581ae640c4df172e746 P 62a1a7bcde864bb6a8a45b0050ab9a26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:24.408805 10516 ts_tablet_manager.cc:1905] T d851db6d5c6f4581ae640c4df172e746 P 62a1a7bcde864bb6a8a45b0050ab9a26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:37:24.410379 10516 ts_tablet_manager.cc:1918] T d851db6d5c6f4581ae640c4df172e746 P 62a1a7bcde864bb6a8a45b0050ab9a26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:37:24.410480 10516 log.cc:1196] T d851db6d5c6f4581ae640c4df172e746 P 62a1a7bcde864bb6a8a45b0050ab9a26: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552619305671-1015-0/minicluster-data/ts-2/wal/wals/d851db6d5c6f4581ae640c4df172e746
I20260812 16:37:24.410789 10516 ts_tablet_manager.cc:1939] T d851db6d5c6f4581ae640c4df172e746 P 62a1a7bcde864bb6a8a45b0050ab9a26: Deleting consensus metadata
I20260812 16:37:24.411335 9635 catalog_manager.cc:4841] TS 62a1a7bcde864bb6a8a45b0050ab9a26 (127.0.253.195:41683): tablet d851db6d5c6f4581ae640c4df172e746 (table test-table [id=cdf6c678138b41a68f3fe1ed9c188a68]) successfully deleted
W20260812 16:37:24.413156 9639 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.253.193:41183 (error 108)
I20260812 16:37:24.413676 10463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:24.413541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:41183 (local address 127.0.0.1:39246)
0812 16:37:24.413596 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:24.413640 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.193:41183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 16:37:24.413792 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 9980
W20260812 16:37:24.413822 9639 catalog_manager.cc:4584] TS 1424b58856cd4370883faebec92cefac (127.0.253.193:41183): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d851db6d5c6f4581ae640c4df172e746: Network error: Client connection negotiation failed: client connection to 127.0.253.193:41183: connect: Connection refused (error 111)
I20260812 16:37:24.442142 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 10224
I20260812 16:37:24.451218 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 9599
2026-08-12T16:37:24Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (921 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260812 16:37:24.478416 1015 mini_oidc.cc:152] Starting JWKS server
I20260812 16:37:24.479821 1015 webserver.cc:460] Webserver started at https://127.0.0.1:32997/ using document root <none> and password file <none>
I20260812 16:37:24.479964 1015 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 16:37:24.480154 1015 webserver.cc:460] Webserver started at http://127.0.0.1:34881/ using document root <none> and password file <none>
2026-08-12T16:37:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:37:24Z Disabled control of system clock
I20260812 16:37:24.489104 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:35991
--webserver_interface=127.0.253.254
--webserver_port=0
--builtin_ntp_servers=127.0.253.212:37475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:35991
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32997/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786552619305671-1015-0/testchainca.cert with env {}
W20260812 16:37:24.573024 10531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:24.573244 10531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:24.573267 10531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:24.574368 10531 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:37:24.574405 10531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:24.574417 10531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:24.574430 10531 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:37:24.574441 10531 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 16:37:24.574453 10531 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 16:37:24.574465 10531 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:32997/jwks.json
I20260812 16:37:24.575979 10531 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:37475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:35991
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:35991
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32997/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786552619305671-1015-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:24.576177 10531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:24.576346 10531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:24.581424 10541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:24.581526 10531 server_base.cc:1034] running on GCE node
W20260812 16:37:24.581593 10546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:24.581458 10540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:24.581959 10531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:24.582262 10531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:24.583424 10531 hybrid_clock.cc:648] HybridClock initialized: now 1786552644583407 us; error 45 us; skew 500 ppm
I20260812 16:37:24.584720 10531 webserver.cc:460] Webserver started at http://127.0.253.254:44029/ using document root <none> and password file <none>
I20260812 16:37:24.584976 10531 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:24.585044 10531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:24.585188 10531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:24.586275 10531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "5434e959bf304719b94aabf0d794e402"
format_stamp: "Formatted at 2026-08-12 16:37:24 on dist-test-slave-w4v5"
server_key: "44c4707a92c31cce8c500b46a54a03eb"
server_key_iv: "6afd195597983fb3cdb3451ab1d9f5f7"
server_key_version: "encryptionkey@0"
I20260812 16:37:24.586642 10531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "5434e959bf304719b94aabf0d794e402"
format_stamp: "Formatted at 2026-08-12 16:37:24 on dist-test-slave-w4v5"
server_key: "44c4707a92c31cce8c500b46a54a03eb"
server_key_iv: "6afd195597983fb3cdb3451ab1d9f5f7"
server_key_version: "encryptionkey@0"
I20260812 16:37:24.588057 10531 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:37:24.588794 10557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:24.588960 10531 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 16:37:24.589067 10531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "5434e959bf304719b94aabf0d794e402"
format_stamp: "Formatted at 2026-08-12 16:37:24 on dist-test-slave-w4v5"
server_key: "44c4707a92c31cce8c500b46a54a03eb"
server_key_iv: "6afd195597983fb3cdb3451ab1d9f5f7"
server_key_version: "encryptionkey@0"
I20260812 16:37:24.589138 10531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:24.609542 10531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:24.609897 10531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:24.610052 10531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:24.614198 10531 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:35991
I20260812 16:37:24.614270 10645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:35991 every 8 connection(s)
I20260812 16:37:24.614627 10531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:37:24.615767 10647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:24.618111 10647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5434e959bf304719b94aabf0d794e402: Bootstrap starting.
I20260812 16:37:24.618676 10647 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5434e959bf304719b94aabf0d794e402: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:24.618920 10647 log.cc:824] T 00000000000000000000000000000000 P 5434e959bf304719b94aabf0d794e402: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:24.619604 10647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5434e959bf304719b94aabf0d794e402: No bootstrap required, opened a new log
I20260812 16:37:24.621114 10647 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5434e959bf304719b94aabf0d794e402 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5434e959bf304719b94aabf0d794e402" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 35991 } }
I20260812 16:37:24.621259 10647 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5434e959bf304719b94aabf0d794e402 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:24.621284 10647 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5434e959bf304719b94aabf0d794e402 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5434e959bf304719b94aabf0d794e402, State: Initialized, Role: FOLLOWER
I20260812 16:37:24.621419 10647 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5434e959bf304719b94aabf0d794e402 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5434e959bf304719b94aabf0d794e402" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 35991 } }
I20260812 16:37:24.621503 10647 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5434e959bf304719b94aabf0d794e402 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:37:24.621527 10647 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5434e959bf304719b94aabf0d794e402 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:37:24.621556 10647 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5434e959bf304719b94aabf0d794e402 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:24.622025 10647 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5434e959bf304719b94aabf0d794e402 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5434e959bf304719b94aabf0d794e402" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 35991 } }
I20260812 16:37:24.622109 10647 leader_election.cc:302] T 00000000000000000000000000000000 P 5434e959bf304719b94aabf0d794e402 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5434e959bf304719b94aabf0d794e402; no voters:
I20260812 16:37:24.622269 10647 leader_election.cc:288] T 00000000000000000000000000000000 P 5434e959bf304719b94aabf0d794e402 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:37:24.622313 10653 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5434e959bf304719b94aabf0d794e402 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:37:24.622452 10653 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5434e959bf304719b94aabf0d794e402 [term 1 LEADER]: Becoming Leader. State: Replica: 5434e959bf304719b94aabf0d794e402, State: Running, Role: LEADER
I20260812 16:37:24.622534 10647 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5434e959bf304719b94aabf0d794e402 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:37:24.622547 10653 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5434e959bf304719b94aabf0d794e402 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5434e959bf304719b94aabf0d794e402" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 35991 } }
I20260812 16:37:24.622974 10658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5434e959bf304719b94aabf0d794e402 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5434e959bf304719b94aabf0d794e402. Latest consensus state: current_term: 1 leader_uuid: "5434e959bf304719b94aabf0d794e402" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5434e959bf304719b94aabf0d794e402" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 35991 } } }
I20260812 16:37:24.623112 10658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5434e959bf304719b94aabf0d794e402 [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:24.623044 10656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5434e959bf304719b94aabf0d794e402 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5434e959bf304719b94aabf0d794e402" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5434e959bf304719b94aabf0d794e402" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 35991 } } }
I20260812 16:37:24.623579 10656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5434e959bf304719b94aabf0d794e402 [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:24.623594 10669 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:37:24.623808 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 10531
I20260812 16:37:24.623925 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:37:24.624116 1015 external_mini_cluster.cc:1428] Setting key 6eee5a50b8e936e4a67a216c8f6029c1
I20260812 16:37:24.624127 10669 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:37:24.625802 10669 catalog_manager.cc:1347] Generated new cluster ID: 6e6ffa744add4fb4bf14cba87348fb53
I20260812 16:37:24.625861 10669 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:37:24.630538 10683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:24.624766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40204 (local address 127.0.253.254:35991)
0812 16:37:24.625024 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:24.625033 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:24.625069 (+ 36us) server_negotiation.cc:407] Connection header received
0812 16:37:24.625203 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:24.625209 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:24.625316 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:24.625467 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:24.625699 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:24.627050 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:24.628226 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:24.628835 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:24.628944 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:24.629674 (+ 730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:24.629678 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:24.629678 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:37:24.629879 (+ 201us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:24.629918 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:37:24.629932 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:24.629994 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:24.630089 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 16:37:24.630173 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":92,"threads_started":1}
I20260812 16:37:24.631417 1015 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:37:24.647199 10669 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:37:24.647727 10669 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:37:24.656211 10669 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5434e959bf304719b94aabf0d794e402: Generated new TSK 0
I20260812 16:37:24.656399 10669 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 16:37:24.679579 10697 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260812 16:37:24.679730 10697 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 16:37:24.679919 10683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:24.664420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40212 (local address 127.0.253.254:35991)
0812 16:37:24.664477 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:24.664479 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:24.664595 (+ 116us) server_negotiation.cc:407] Connection header received
0812 16:37:24.664666 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:24.664667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:24.664679 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:24.664757 (+ 78us) server_negotiation.cc:226] Negotiated authn=JWT
0812 16:37:24.664875 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:24.665671 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:24.666416 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:24.666718 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:24.666797 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:24.666956 (+ 159us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 16:37:24.679709 (+ 12753us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32997/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 16:37:24.679850 (+ 141us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40212: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32997/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":34}
W20260812 16:37:24.679975 10683 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40212: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32997/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260812 16:37:24.680532 10694 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:24.664337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:35991 (local address 127.0.0.1:40212)
0812 16:37:24.664533 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:24.664542 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:24.664597 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:24.664771 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:24.664771 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:24.664773 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0812 16:37:24.664822 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:24.664824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:24.665751 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:24.665752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:24.666341 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:24.666343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:24.666803 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:24.666804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:24.666843 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:24.666861 (+ 18us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 16:37:24.679888 (+ 13027us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32997/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 16:37:24.680100 (+ 212us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.253.254:35991: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32997/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":103,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.0.253.254:35991: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32997/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260812 16:37:24.681061 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 10531
2026-08-12T16:37:24Z chronyd exiting
I20260812 16:37:24.724624 1015 test_util.cc:175] -----------------------------------------------
I20260812 16:37:24.724732 1015 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786552619305671-1015-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (247 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260812 16:37:24.725550 1015 mini_oidc.cc:152] Starting JWKS server
I20260812 16:37:24.726253 1015 webserver.cc:460] Webserver started at https://127.0.0.1:33199/ using document root <none> and password file <none>
I20260812 16:37:24.726317 1015 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 16:37:24.726495 1015 webserver.cc:460] Webserver started at http://127.0.0.1:41821/ using document root <none> and password file <none>
2026-08-12T16:37:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:37:24Z Disabled control of system clock
I20260812 16:37:24.735378 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:35799
--webserver_interface=127.0.253.254
--webserver_port=0
--builtin_ntp_servers=127.0.253.212:35615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:35799
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33199/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786552619305671-1015-0/testchainca.cert with env {}
W20260812 16:37:24.824481 10704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:24.824702 10704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:24.824723 10704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:24.826005 10704 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:37:24.826051 10704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:24.826064 10704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:24.826077 10704 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:37:24.826089 10704 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 16:37:24.826102 10704 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 16:37:24.826136 10704 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33199/jwks.json
I20260812 16:37:24.827826 10704 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:35615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:35799
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:35799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33199/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786552619305671-1015-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:24.828084 10704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:24.828317 10704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:24.833492 10718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:24.833496 10711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:24.833540 10712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:24.833777 10704 server_base.cc:1034] running on GCE node
I20260812 16:37:24.834040 10704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:24.834345 10704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:24.835505 10704 hybrid_clock.cc:648] HybridClock initialized: now 1786552644835491 us; error 31 us; skew 500 ppm
I20260812 16:37:24.836781 10704 webserver.cc:460] Webserver started at http://127.0.253.254:42485/ using document root <none> and password file <none>
I20260812 16:37:24.837044 10704 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:24.837091 10704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:24.837258 10704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:24.838356 10704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "4927b103ce984bbd95f53d1ebe878a0f"
format_stamp: "Formatted at 2026-08-12 16:37:24 on dist-test-slave-w4v5"
server_key: "c2da75e3d09c4fd35f8b707d4e3cdf3a"
server_key_iv: "14685c8911da03dfb3925ce0afabe897"
server_key_version: "encryptionkey@0"
I20260812 16:37:24.838719 10704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "4927b103ce984bbd95f53d1ebe878a0f"
format_stamp: "Formatted at 2026-08-12 16:37:24 on dist-test-slave-w4v5"
server_key: "c2da75e3d09c4fd35f8b707d4e3cdf3a"
server_key_iv: "14685c8911da03dfb3925ce0afabe897"
server_key_version: "encryptionkey@0"
I20260812 16:37:24.840103 10704 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:37:24.840847 10732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:24.841012 10704 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 16:37:24.841112 10704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "4927b103ce984bbd95f53d1ebe878a0f"
format_stamp: "Formatted at 2026-08-12 16:37:24 on dist-test-slave-w4v5"
server_key: "c2da75e3d09c4fd35f8b707d4e3cdf3a"
server_key_iv: "14685c8911da03dfb3925ce0afabe897"
server_key_version: "encryptionkey@0"
I20260812 16:37:24.841184 10704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:24.851703 10704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:24.851994 10704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:24.852135 10704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:24.856434 10704 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:35799
I20260812 16:37:24.856495 10820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:35799 every 8 connection(s)
I20260812 16:37:24.856824 10704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:37:24.857810 10821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:24.859879 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 10704
I20260812 16:37:24.860000 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:37:24.860195 1015 external_mini_cluster.cc:1428] Setting key e8f05fc9fab665f975a15a576416f510
I20260812 16:37:24.860471 10821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4927b103ce984bbd95f53d1ebe878a0f: Bootstrap starting.
I20260812 16:37:24.861107 10821 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4927b103ce984bbd95f53d1ebe878a0f: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:24.861404 10821 log.cc:824] T 00000000000000000000000000000000 P 4927b103ce984bbd95f53d1ebe878a0f: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:24.862260 10821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4927b103ce984bbd95f53d1ebe878a0f: No bootstrap required, opened a new log
I20260812 16:37:24.864768 10821 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4927b103ce984bbd95f53d1ebe878a0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4927b103ce984bbd95f53d1ebe878a0f" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 35799 } }
I20260812 16:37:24.864897 10821 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4927b103ce984bbd95f53d1ebe878a0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:24.864920 10821 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4927b103ce984bbd95f53d1ebe878a0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4927b103ce984bbd95f53d1ebe878a0f, State: Initialized, Role: FOLLOWER
I20260812 16:37:24.865070 10821 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4927b103ce984bbd95f53d1ebe878a0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4927b103ce984bbd95f53d1ebe878a0f" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 35799 } }
I20260812 16:37:24.865170 10821 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4927b103ce984bbd95f53d1ebe878a0f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:37:24.865216 10821 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4927b103ce984bbd95f53d1ebe878a0f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:37:24.865270 10821 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4927b103ce984bbd95f53d1ebe878a0f [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:24.865989 10821 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4927b103ce984bbd95f53d1ebe878a0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4927b103ce984bbd95f53d1ebe878a0f" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 35799 } }
I20260812 16:37:24.866106 10821 leader_election.cc:302] T 00000000000000000000000000000000 P 4927b103ce984bbd95f53d1ebe878a0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4927b103ce984bbd95f53d1ebe878a0f; no voters:
I20260812 16:37:24.866291 10821 leader_election.cc:288] T 00000000000000000000000000000000 P 4927b103ce984bbd95f53d1ebe878a0f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:37:24.866317 10834 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4927b103ce984bbd95f53d1ebe878a0f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:37:24.866451 10834 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4927b103ce984bbd95f53d1ebe878a0f [term 1 LEADER]: Becoming Leader. State: Replica: 4927b103ce984bbd95f53d1ebe878a0f, State: Running, Role: LEADER
I20260812 16:37:24.866518 10830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:24.860703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37352 (local address 127.0.253.254:35799)
0812 16:37:24.860906 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:24.860919 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:24.860943 (+ 24us) server_negotiation.cc:407] Connection header received
0812 16:37:24.861431 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:24.861438 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:24.861534 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:24.861697 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:24.861873 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:24.863319 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:24.864481 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:24.864882 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:24.865028 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:24.865723 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:24.865727 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:24.865727 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:37:24.865893 (+ 166us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:24.865925 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:37:24.865935 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:24.865986 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:24.866119 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 16:37:24.866210 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":92,"threads_started":1}
I20260812 16:37:24.866575 10821 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4927b103ce984bbd95f53d1ebe878a0f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:37:24.866535 10834 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4927b103ce984bbd95f53d1ebe878a0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4927b103ce984bbd95f53d1ebe878a0f" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 35799 } }
I20260812 16:37:24.866917 10837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4927b103ce984bbd95f53d1ebe878a0f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4927b103ce984bbd95f53d1ebe878a0f. Latest consensus state: current_term: 1 leader_uuid: "4927b103ce984bbd95f53d1ebe878a0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4927b103ce984bbd95f53d1ebe878a0f" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 35799 } } }
I20260812 16:37:24.867060 10837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4927b103ce984bbd95f53d1ebe878a0f [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:24.866925 10836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4927b103ce984bbd95f53d1ebe878a0f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4927b103ce984bbd95f53d1ebe878a0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4927b103ce984bbd95f53d1ebe878a0f" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 35799 } } }
I20260812 16:37:24.867489 1015 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:37:24.867805 10849 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:37:24.868270 10836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4927b103ce984bbd95f53d1ebe878a0f [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:24.868374 10849 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:37:24.869853 10849 catalog_manager.cc:1347] Generated new cluster ID: ec4a28dc565347d78356fd843905e9d1
I20260812 16:37:24.869907 10849 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:37:24.883598 10849 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:37:24.884282 10849 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:37:24.888952 10849 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4927b103ce984bbd95f53d1ebe878a0f: Generated new TSK 0
I20260812 16:37:24.889124 10849 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 16:37:24.913195 10889 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260812 16:37:24.913295 10889 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 16:37:24.913650 10830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:24.895396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37354 (local address 127.0.253.254:35799)
0812 16:37:24.895458 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:24.895460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:24.895585 (+ 125us) server_negotiation.cc:407] Connection header received
0812 16:37:24.895662 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:24.895663 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:24.895673 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:24.895733 (+ 60us) server_negotiation.cc:226] Negotiated authn=JWT
0812 16:37:24.895903 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:24.896384 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:24.897546 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:24.897745 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:24.897804 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:24.897995 (+ 191us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 16:37:24.913411 (+ 15416us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33199/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 16:37:24.913574 (+ 163us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37354: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33199/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":35}
W20260812 16:37:24.913728 10830 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37354: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33199/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260812 16:37:24.913765 10888 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:24.895325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:35799 (local address 127.0.0.1:37354)
0812 16:37:24.895510 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:24.895523 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:24.895592 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:24.895761 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:24.895762 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:24.895764 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0812 16:37:24.895828 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:24.895831 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:24.896472 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:24.896474 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:24.897446 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:24.897449 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:24.897822 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:24.897823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:24.897891 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:24.897897 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 16:37:24.913547 (+ 15650us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33199/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 16:37:24.913665 (+ 118us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.253.254:35799: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33199/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":116,"threads_started":1}
I20260812 16:37:24.914057 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 10704
2026-08-12T16:37:24Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (249 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260812 16:37:24.974758 1015 mini_oidc.cc:152] Starting JWKS server
I20260812 16:37:24.975566 1015 webserver.cc:460] Webserver started at https://127.0.0.1:35077/ using document root <none> and password file <none>
I20260812 16:37:24.975633 1015 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 16:37:24.975814 1015 webserver.cc:460] Webserver started at http://127.0.0.1:43199/ using document root <none> and password file <none>
2026-08-12T16:37:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:37:24Z Disabled control of system clock
I20260812 16:37:24.985126 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:37567
--webserver_interface=127.0.253.254
--webserver_port=0
--builtin_ntp_servers=127.0.253.212:35055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:37567
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35077/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 16:37:25.070456 10897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:25.070695 10897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:25.070719 10897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:25.071969 10897 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:37:25.072014 10897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:25.072027 10897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:25.072041 10897 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:37:25.072088 10897 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 16:37:25.072106 10897 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 16:37:25.072129 10897 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35077/jwks.json
I20260812 16:37:25.073510 10897 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:35055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:37567
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:37567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35077/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:25.073781 10897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:25.074080 10897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:25.079064 10905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:25.079111 10908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:25.079109 10904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:25.079376 10897 server_base.cc:1034] running on GCE node
I20260812 16:37:25.079643 10897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:25.079950 10897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:25.081113 10897 hybrid_clock.cc:648] HybridClock initialized: now 1786552645081099 us; error 41 us; skew 500 ppm
I20260812 16:37:25.082396 10897 webserver.cc:460] Webserver started at http://127.0.253.254:35593/ using document root <none> and password file <none>
I20260812 16:37:25.082639 10897 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:25.082691 10897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:25.082849 10897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:25.083972 10897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "8d6a1cbe31674aebab451096f19f8d41"
format_stamp: "Formatted at 2026-08-12 16:37:25 on dist-test-slave-w4v5"
server_key: "b7d14456cda669aee116eaa1bb617d98"
server_key_iv: "8942bdcc73d4be7ac8125fa65fec2d57"
server_key_version: "encryptionkey@0"
I20260812 16:37:25.084352 10897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "8d6a1cbe31674aebab451096f19f8d41"
format_stamp: "Formatted at 2026-08-12 16:37:25 on dist-test-slave-w4v5"
server_key: "b7d14456cda669aee116eaa1bb617d98"
server_key_iv: "8942bdcc73d4be7ac8125fa65fec2d57"
server_key_version: "encryptionkey@0"
I20260812 16:37:25.085731 10897 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:37:25.086444 10915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:25.086603 10897 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:37:25.086719 10897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "8d6a1cbe31674aebab451096f19f8d41"
format_stamp: "Formatted at 2026-08-12 16:37:25 on dist-test-slave-w4v5"
server_key: "b7d14456cda669aee116eaa1bb617d98"
server_key_iv: "8942bdcc73d4be7ac8125fa65fec2d57"
server_key_version: "encryptionkey@0"
I20260812 16:37:25.086798 10897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:25.097640 10897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:25.097957 10897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:25.098102 10897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:25.102170 10897 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:37567
I20260812 16:37:25.102201 11016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:37567 every 8 connection(s)
I20260812 16:37:25.102547 10897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:37:25.103279 11018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:25.106154 11018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d6a1cbe31674aebab451096f19f8d41: Bootstrap starting.
I20260812 16:37:25.106786 11018 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d6a1cbe31674aebab451096f19f8d41: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:25.107122 11018 log.cc:824] T 00000000000000000000000000000000 P 8d6a1cbe31674aebab451096f19f8d41: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:25.107836 11018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d6a1cbe31674aebab451096f19f8d41: No bootstrap required, opened a new log
I20260812 16:37:25.109427 11018 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d6a1cbe31674aebab451096f19f8d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d6a1cbe31674aebab451096f19f8d41" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 37567 } }
I20260812 16:37:25.109565 11018 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d6a1cbe31674aebab451096f19f8d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:25.109618 11018 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d6a1cbe31674aebab451096f19f8d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d6a1cbe31674aebab451096f19f8d41, State: Initialized, Role: FOLLOWER
I20260812 16:37:25.109804 11018 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d6a1cbe31674aebab451096f19f8d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d6a1cbe31674aebab451096f19f8d41" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 37567 } }
I20260812 16:37:25.109905 11018 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8d6a1cbe31674aebab451096f19f8d41 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:37:25.109943 11018 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d6a1cbe31674aebab451096f19f8d41 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:37:25.109995 11018 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d6a1cbe31674aebab451096f19f8d41 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:25.109984 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 10897
I20260812 16:37:25.110097 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:37:25.110287 1015 external_mini_cluster.cc:1428] Setting key 9dfb6e7ce78c4384cb3cc08b914b57b2
I20260812 16:37:25.110796 11018 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d6a1cbe31674aebab451096f19f8d41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d6a1cbe31674aebab451096f19f8d41" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 37567 } }
I20260812 16:37:25.110915 11018 leader_election.cc:302] T 00000000000000000000000000000000 P 8d6a1cbe31674aebab451096f19f8d41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d6a1cbe31674aebab451096f19f8d41; no voters:
I20260812 16:37:25.111100 11018 leader_election.cc:288] T 00000000000000000000000000000000 P 8d6a1cbe31674aebab451096f19f8d41 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:37:25.111122 11025 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d6a1cbe31674aebab451096f19f8d41 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:37:25.111241 11025 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d6a1cbe31674aebab451096f19f8d41 [term 1 LEADER]: Becoming Leader. State: Replica: 8d6a1cbe31674aebab451096f19f8d41, State: Running, Role: LEADER
I20260812 16:37:25.111330 11025 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d6a1cbe31674aebab451096f19f8d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d6a1cbe31674aebab451096f19f8d41" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 37567 } }
I20260812 16:37:25.111389 11018 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d6a1cbe31674aebab451096f19f8d41 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:37:25.111672 11028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d6a1cbe31674aebab451096f19f8d41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8d6a1cbe31674aebab451096f19f8d41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d6a1cbe31674aebab451096f19f8d41" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 37567 } } }
I20260812 16:37:25.111765 11028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d6a1cbe31674aebab451096f19f8d41 [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:25.112104 11031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d6a1cbe31674aebab451096f19f8d41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d6a1cbe31674aebab451096f19f8d41. Latest consensus state: current_term: 1 leader_uuid: "8d6a1cbe31674aebab451096f19f8d41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d6a1cbe31674aebab451096f19f8d41" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 37567 } } }
I20260812 16:37:25.112183 11031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d6a1cbe31674aebab451096f19f8d41 [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:25.112299 11036 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:37:25.112802 11036 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:37:25.114730 11036 catalog_manager.cc:1347] Generated new cluster ID: 691a0462116749aaa6e7e7eb329bbeb6
I20260812 16:37:25.114782 11036 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:37:25.116374 11024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:25.110790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37844 (local address 127.0.253.254:37567)
0812 16:37:25.111023 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:25.111032 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:25.111069 (+ 37us) server_negotiation.cc:407] Connection header received
0812 16:37:25.111213 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:25.111221 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:25.111297 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:25.111726 (+ 429us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:25.111829 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:25.113256 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:25.114250 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:25.114866 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:25.114988 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:25.115710 (+ 722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:25.115714 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:25.115714 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:37:25.115893 (+ 179us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:25.115927 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:37:25.115937 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:25.116010 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:25.116087 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 16:37:25.116169 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":100,"threads_started":1}
I20260812 16:37:25.117273 1015 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:37:25.131057 11036 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:37:25.131633 11036 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:37:25.134549 11036 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8d6a1cbe31674aebab451096f19f8d41: Generated new TSK 0
I20260812 16:37:25.134737 11036 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:37:25.143815 11024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:25.140149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37848 (local address 127.0.253.254:37567)
0812 16:37:25.140195 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:25.140197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:25.140304 (+ 107us) server_negotiation.cc:407] Connection header received
0812 16:37:25.140381 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:25.140382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:25.140394 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:25.140454 (+ 60us) server_negotiation.cc:226] Negotiated authn=JWT
0812 16:37:25.140619 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:25.141125 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:25.142274 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:25.142472 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:25.142522 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:25.142724 (+ 202us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 16:37:25.143556 (+ 832us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35077/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 16:37:25.143742 (+ 186us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37848: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35077/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":24}
W20260812 16:37:25.143852 11065 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:25.140045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:37567 (local address 127.0.0.1:37848)
0812 16:37:25.140230 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:25.140242 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:25.140312 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:25.140477 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:25.140478 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:25.140481 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0812 16:37:25.140557 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:25.140560 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:25.141227 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:25.141228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:25.142195 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:25.142198 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:25.142554 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:25.142555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:25.142622 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:25.142628 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 16:37:25.143666 (+ 1038us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35077/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 16:37:25.143760 (+ 94us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.253.254:37567: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35077/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":95,"threads_started":1}
W20260812 16:37:25.143886 11024 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37848: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35077/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260812 16:37:25.144186 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 10897
E20260812 16:37:25.162809 11072 webserver.cc:533] Webserver: sslize failed:
E20260812 16:37:25.162891 11072 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-12T16:37:25Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (219 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-12T16:37:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:37:25Z Disabled control of system clock
I20260812 16:37:25.203545 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:46735
--webserver_interface=127.0.253.254
--webserver_port=0
--builtin_ntp_servers=127.0.253.212:36107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:46735
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786552619305671-1015-0/testchainca.cert with env {}
W20260812 16:37:25.287168 11079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:25.287390 11079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:25.287412 11079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:25.289773 11079 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:37:25.289850 11079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:25.289876 11079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:25.289901 11079 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:37:25.289922 11079 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 16:37:25.289944 11079 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 16:37:25.289969 11079 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260812 16:37:25.291764 11079 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:36107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:46735
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:46735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786552619305671-1015-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11079
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:25.291987 11079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:25.292176 11079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:25.297312 11093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:25.297304 11090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:25.297300 11085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:25.297468 11079 server_base.cc:1034] running on GCE node
I20260812 16:37:25.297763 11079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:25.298064 11079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:25.299202 11079 hybrid_clock.cc:648] HybridClock initialized: now 1786552645299189 us; error 41 us; skew 500 ppm
I20260812 16:37:25.300493 11079 webserver.cc:460] Webserver started at http://127.0.253.254:46203/ using document root <none> and password file <none>
I20260812 16:37:25.300776 11079 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:25.300845 11079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:25.300990 11079 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:25.302098 11079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "8f727178942e425aa9d08bf7fcaa793d"
format_stamp: "Formatted at 2026-08-12 16:37:25 on dist-test-slave-w4v5"
server_key: "b306aab29a9882024802712057dad92e"
server_key_iv: "d2c3bac2acd53fc6ce24fe44a5892ed2"
server_key_version: "encryptionkey@0"
I20260812 16:37:25.302461 11079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "8f727178942e425aa9d08bf7fcaa793d"
format_stamp: "Formatted at 2026-08-12 16:37:25 on dist-test-slave-w4v5"
server_key: "b306aab29a9882024802712057dad92e"
server_key_iv: "d2c3bac2acd53fc6ce24fe44a5892ed2"
server_key_version: "encryptionkey@0"
I20260812 16:37:25.303880 11079 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:37:25.304630 11101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:25.304811 11079 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:37:25.304914 11079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "8f727178942e425aa9d08bf7fcaa793d"
format_stamp: "Formatted at 2026-08-12 16:37:25 on dist-test-slave-w4v5"
server_key: "b306aab29a9882024802712057dad92e"
server_key_iv: "d2c3bac2acd53fc6ce24fe44a5892ed2"
server_key_version: "encryptionkey@0"
I20260812 16:37:25.304987 11079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:25.326395 11079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:25.326758 11079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:25.326915 11079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:25.331037 11079 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:46735
I20260812 16:37:25.331066 11199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:46735 every 8 connection(s)
I20260812 16:37:25.331494 11079 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:37:25.332481 11200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:25.334728 11200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f727178942e425aa9d08bf7fcaa793d: Bootstrap starting.
I20260812 16:37:25.335300 11200 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f727178942e425aa9d08bf7fcaa793d: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:25.335541 11200 log.cc:824] T 00000000000000000000000000000000 P 8f727178942e425aa9d08bf7fcaa793d: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:25.336261 11200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f727178942e425aa9d08bf7fcaa793d: No bootstrap required, opened a new log
I20260812 16:37:25.337854 11200 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f727178942e425aa9d08bf7fcaa793d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f727178942e425aa9d08bf7fcaa793d" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 46735 } }
I20260812 16:37:25.337992 11200 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f727178942e425aa9d08bf7fcaa793d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:25.338014 11200 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f727178942e425aa9d08bf7fcaa793d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f727178942e425aa9d08bf7fcaa793d, State: Initialized, Role: FOLLOWER
I20260812 16:37:25.338161 11200 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f727178942e425aa9d08bf7fcaa793d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f727178942e425aa9d08bf7fcaa793d" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 46735 } }
I20260812 16:37:25.338243 11200 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f727178942e425aa9d08bf7fcaa793d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:37:25.338289 11200 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f727178942e425aa9d08bf7fcaa793d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:37:25.338344 11200 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f727178942e425aa9d08bf7fcaa793d [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:25.338485 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 11079
I20260812 16:37:25.338593 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:37:25.338791 1015 external_mini_cluster.cc:1428] Setting key 992c8098b0b2a82862285b0a7df0f304
I20260812 16:37:25.339066 11200 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f727178942e425aa9d08bf7fcaa793d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f727178942e425aa9d08bf7fcaa793d" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 46735 } }
I20260812 16:37:25.339233 11200 leader_election.cc:302] T 00000000000000000000000000000000 P 8f727178942e425aa9d08bf7fcaa793d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f727178942e425aa9d08bf7fcaa793d; no voters:
I20260812 16:37:25.339406 11200 leader_election.cc:288] T 00000000000000000000000000000000 P 8f727178942e425aa9d08bf7fcaa793d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:37:25.339591 11209 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f727178942e425aa9d08bf7fcaa793d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:37:25.339677 11200 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f727178942e425aa9d08bf7fcaa793d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:37:25.339740 11209 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f727178942e425aa9d08bf7fcaa793d [term 1 LEADER]: Becoming Leader. State: Replica: 8f727178942e425aa9d08bf7fcaa793d, State: Running, Role: LEADER
I20260812 16:37:25.339836 11209 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f727178942e425aa9d08bf7fcaa793d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f727178942e425aa9d08bf7fcaa793d" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 46735 } }
I20260812 16:37:25.340229 11209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f727178942e425aa9d08bf7fcaa793d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f727178942e425aa9d08bf7fcaa793d. Latest consensus state: current_term: 1 leader_uuid: "8f727178942e425aa9d08bf7fcaa793d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f727178942e425aa9d08bf7fcaa793d" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 46735 } } }
I20260812 16:37:25.340318 11209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f727178942e425aa9d08bf7fcaa793d [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:25.340478 11204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f727178942e425aa9d08bf7fcaa793d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f727178942e425aa9d08bf7fcaa793d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f727178942e425aa9d08bf7fcaa793d" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 46735 } } }
I20260812 16:37:25.340597 11204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f727178942e425aa9d08bf7fcaa793d [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:25.340721 11221 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:37:25.341233 11221 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:37:25.342808 11221 catalog_manager.cc:1347] Generated new cluster ID: 4da57ddc1bb64f85b280a7026e70ce38
I20260812 16:37:25.342866 11221 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:37:25.345098 11206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:25.339288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54624 (local address 127.0.253.254:46735)
0812 16:37:25.339556 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:25.339566 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:25.339600 (+ 34us) server_negotiation.cc:407] Connection header received
0812 16:37:25.339730 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:25.339738 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:25.339809 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:25.339970 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:25.340241 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:25.342210 (+ 1969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:25.343181 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:25.343629 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:25.343759 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:25.344428 (+ 669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:25.344432 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:25.344433 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:37:25.344618 (+ 185us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:25.344653 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:37:25.344663 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:25.344740 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:25.344811 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 16:37:25.344893 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":94,"threads_started":1}
I20260812 16:37:25.345956 1015 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:37:25.357862 11221 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:37:25.358366 11221 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:37:25.361639 11221 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f727178942e425aa9d08bf7fcaa793d: Generated new TSK 0
I20260812 16:37:25.361802 11221 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:37:26.023563 11206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:25.370538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54628 (local address 127.0.253.254:46735)
0812 16:37:25.370601 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:25.370603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:25.370720 (+ 117us) server_negotiation.cc:407] Connection header received
0812 16:37:25.370799 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:25.370799 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:25.370811 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:25.370871 (+ 60us) server_negotiation.cc:226] Negotiated authn=JWT
0812 16:37:25.371031 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:25.371611 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:25.372773 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:25.372982 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:25.373040 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:25.373224 (+ 184us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 16:37:26.023326 (+650102us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 16:37:26.023498 (+ 172us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54628: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":37}
W20260812 16:37:26.023681 11206 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54628: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260812 16:37:26.023761 11243 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:25.370470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:46735 (local address 127.0.0.1:54628)
0812 16:37:25.370646 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:25.370658 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:25.370729 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:25.370887 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:25.370888 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:25.370890 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0812 16:37:25.370954 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:25.370956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:25.371698 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:25.371700 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:25.372669 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:25.372672 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:25.373070 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:25.373071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:25.373132 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:25.373138 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 16:37:26.023492 (+650354us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 16:37:26.023639 (+ 147us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.253.254:46735: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":110,"threads_started":1}
I20260812 16:37:26.024238 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 11079
2026-08-12T16:37:26Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (857 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (97 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (92 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:37:26 dist-test-slave-w4v5 krb5kdc[11267](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:37:26 dist-test-slave-w4v5 krb5kdc[11267](info): set up 2 sockets
Aug 12 16:37:26 dist-test-slave-w4v5 krb5kdc[11267](info): commencing operation
krb5kdc: starting...
W20260812 16:37:27.564327 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.322s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260812 16:37:27.686373 1015 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.122s user 0.000s sys 0.002s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260812 16:37:27.908545 1015 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.222s user 0.000s sys 0.002s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260812 16:37:28.042364 1015 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.134s user 0.000s sys 0.002s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/krb5kdc/test-user.keytab.
W20260812 16:37:28.728266 1015 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.686s user 0.000s sys 0.002s
Aug 12 16:37:28 dist-test-slave-w4v5 krb5kdc[11267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:37:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:37:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
W20260812 16:37:29.078171 1015 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.0.253.254: real 0.312s user 0.000s sys 0.005s
I20260812 16:37:29.078585 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:34093
--webserver_interface=127.0.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:36569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:34093
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:29.162978 11285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:29.163223 11285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:29.163246 11285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:29.164458 11285 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:37:29.164497 11285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:29.164512 11285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:29.164525 11285 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:37:29.164537 11285 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:37:29.165948 11285 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:36569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:34093
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:34093
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:29.166142 11285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:29.166316 11285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:29.171320 11295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:29.171299 11297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:29.171446 11285 server_base.cc:1034] running on GCE node
W20260812 16:37:29.171304 11304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:29.171825 11285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:29.172083 11285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:29.173219 11285 hybrid_clock.cc:648] HybridClock initialized: now 1786552649173193 us; error 52 us; skew 500 ppm
Aug 12 16:37:29 dist-test-slave-w4v5 krb5kdc[11267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552649, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:29.175712 11285 init.cc:377] Logged in from keytab as kudu/127.0.253.254@KRBTEST.COM (short username kudu)
I20260812 16:37:29.176126 11285 webserver.cc:460] Webserver started at http://127.0.253.254:41611/ using document root <none> and password file <none>
I20260812 16:37:29.176308 11285 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:29.176347 11285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:29.176436 11285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:29.177453 11285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "bd30f975d0e74bdb8a6302e3f703b5bb"
format_stamp: "Formatted at 2026-08-12 16:37:29 on dist-test-slave-w4v5"
server_key: "8bae3489c534099c8bc46261c40145b3"
server_key_iv: "4ac14c6aaefda3328d7b596b5bb35774"
server_key_version: "encryptionkey@0"
I20260812 16:37:29.177769 11285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "bd30f975d0e74bdb8a6302e3f703b5bb"
format_stamp: "Formatted at 2026-08-12 16:37:29 on dist-test-slave-w4v5"
server_key: "8bae3489c534099c8bc46261c40145b3"
server_key_iv: "4ac14c6aaefda3328d7b596b5bb35774"
server_key_version: "encryptionkey@0"
I20260812 16:37:29.179113 11285 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 16:37:29.179893 11316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:29.180076 11285 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:29.180171 11285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "bd30f975d0e74bdb8a6302e3f703b5bb"
format_stamp: "Formatted at 2026-08-12 16:37:29 on dist-test-slave-w4v5"
server_key: "8bae3489c534099c8bc46261c40145b3"
server_key_iv: "4ac14c6aaefda3328d7b596b5bb35774"
server_key_version: "encryptionkey@0"
I20260812 16:37:29.180243 11285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:29.194427 11285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:29.196929 11285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:29.197093 11285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:29.201105 11285 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:34093
I20260812 16:37:29.201162 11409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:34093 every 8 connection(s)
I20260812 16:37:29.201520 11285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:37:29.202473 11410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:29.204487 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 11285
I20260812 16:37:29.204592 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:37:29.204813 1015 external_mini_cluster.cc:1428] Setting key a1841ea3ef1e23b6a1ee484bee2b6f99
I20260812 16:37:29.205348 11410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd30f975d0e74bdb8a6302e3f703b5bb: Bootstrap starting.
I20260812 16:37:29.206017 11410 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd30f975d0e74bdb8a6302e3f703b5bb: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:29.206372 11410 log.cc:824] T 00000000000000000000000000000000 P bd30f975d0e74bdb8a6302e3f703b5bb: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:29.207401 11410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd30f975d0e74bdb8a6302e3f703b5bb: No bootstrap required, opened a new log
I20260812 16:37:29.209993 11410 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd30f975d0e74bdb8a6302e3f703b5bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd30f975d0e74bdb8a6302e3f703b5bb" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 34093 } }
I20260812 16:37:29.210160 11410 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd30f975d0e74bdb8a6302e3f703b5bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:29.210207 11410 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd30f975d0e74bdb8a6302e3f703b5bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd30f975d0e74bdb8a6302e3f703b5bb, State: Initialized, Role: FOLLOWER
I20260812 16:37:29.210407 11410 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd30f975d0e74bdb8a6302e3f703b5bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd30f975d0e74bdb8a6302e3f703b5bb" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 34093 } }
I20260812 16:37:29.210532 11410 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd30f975d0e74bdb8a6302e3f703b5bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:37:29.210574 11410 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd30f975d0e74bdb8a6302e3f703b5bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:37:29.210634 11410 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd30f975d0e74bdb8a6302e3f703b5bb [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:29.211472 11410 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd30f975d0e74bdb8a6302e3f703b5bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd30f975d0e74bdb8a6302e3f703b5bb" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 34093 } }
I20260812 16:37:29.211608 11410 leader_election.cc:302] T 00000000000000000000000000000000 P bd30f975d0e74bdb8a6302e3f703b5bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd30f975d0e74bdb8a6302e3f703b5bb; no voters:
I20260812 16:37:29.211810 11410 leader_election.cc:288] T 00000000000000000000000000000000 P bd30f975d0e74bdb8a6302e3f703b5bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:37:29.211865 11419 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd30f975d0e74bdb8a6302e3f703b5bb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:37:29.212055 11419 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd30f975d0e74bdb8a6302e3f703b5bb [term 1 LEADER]: Becoming Leader. State: Replica: bd30f975d0e74bdb8a6302e3f703b5bb, State: Running, Role: LEADER
I20260812 16:37:29.212127 11410 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd30f975d0e74bdb8a6302e3f703b5bb [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:37:29.212186 11419 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd30f975d0e74bdb8a6302e3f703b5bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd30f975d0e74bdb8a6302e3f703b5bb" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 34093 } }
I20260812 16:37:29.212567 11419 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd30f975d0e74bdb8a6302e3f703b5bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd30f975d0e74bdb8a6302e3f703b5bb. Latest consensus state: current_term: 1 leader_uuid: "bd30f975d0e74bdb8a6302e3f703b5bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd30f975d0e74bdb8a6302e3f703b5bb" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 34093 } } }
Aug 12 16:37:29 dist-test-slave-w4v5 krb5kdc[11267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:29.212883 11419 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd30f975d0e74bdb8a6302e3f703b5bb [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:29.212749 11422 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd30f975d0e74bdb8a6302e3f703b5bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd30f975d0e74bdb8a6302e3f703b5bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd30f975d0e74bdb8a6302e3f703b5bb" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 34093 } } }
I20260812 16:37:29.213008 11422 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd30f975d0e74bdb8a6302e3f703b5bb [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:29.213542 11440 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:37:29.214139 11440 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:37:29.215766 11440 catalog_manager.cc:1347] Generated new cluster ID: f350757651454b158d22c2e565a482ca
I20260812 16:37:29.215819 11440 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:37:29.221737 11414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:29.205368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55852 (local address 127.0.253.254:34093)
0812 16:37:29.205604 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:29.205611 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:29.205638 (+ 27us) server_negotiation.cc:407] Connection header received
0812 16:37:29.206140 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:29.206154 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:29.206207 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:29.206339 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:29.207185 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:29.208626 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:29.209781 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:29.210202 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:29.210312 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:29.213433 (+ 3121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:29.213438 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:29.213439 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:29.213452 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:29.219904 (+ 6452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:29.220293 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:29.220296 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:29.220297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:29.220395 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:29.220605 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:29.220606 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:29.220606 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:29.220857 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:29.221043 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:29.221475 (+ 432us) server_negotiation.cc:299] Negotiation successful
0812 16:37:29.221591 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":86,"threads_started":1}
I20260812 16:37:29.240088 11440 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:37:29.240670 11440 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:37:29.251477 11440 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd30f975d0e74bdb8a6302e3f703b5bb: Generated new TSK 0
I20260812 16:37:29.251708 11440 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab.
Entry for principal kudu/127.0.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab.
I20260812 16:37:29.287025 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.193:0
--local_ip_for_outbound_sockets=127.0.253.193
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:34093
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:36569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:29.372583 11459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:29.372819 11459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:29.372859 11459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:29.374090 11459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:29.374162 11459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:29.374204 11459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.193
I20260812 16:37:29.375733 11459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:36569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:34093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.193
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:29.376003 11459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:29.376255 11459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:29.381554 11473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:29.381567 11470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:29.381735 11459 server_base.cc:1034] running on GCE node
W20260812 16:37:29.381824 11469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:29.382105 11459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:29.382400 11459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:29.383553 11459 hybrid_clock.cc:648] HybridClock initialized: now 1786552649383534 us; error 40 us; skew 500 ppm
Aug 12 16:37:29 dist-test-slave-w4v5 krb5kdc[11267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552649, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:29.386468 11459 init.cc:377] Logged in from keytab as kudu/127.0.253.193@KRBTEST.COM (short username kudu)
I20260812 16:37:29.386945 11459 webserver.cc:460] Webserver started at http://127.0.253.193:44779/ using document root <none> and password file <none>
I20260812 16:37:29.387202 11459 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:29.387261 11459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:29.387398 11459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:29.388505 11459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-0/data/instance:
uuid: "1150defc97664bd989f0eee19b7140aa"
format_stamp: "Formatted at 2026-08-12 16:37:29 on dist-test-slave-w4v5"
server_key: "bea9e3f51e111ecec871ed0ed804f814"
server_key_iv: "53bbff95c620cf147ea6491c0489ab56"
server_key_version: "encryptionkey@0"
I20260812 16:37:29.388890 11459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance:
uuid: "1150defc97664bd989f0eee19b7140aa"
format_stamp: "Formatted at 2026-08-12 16:37:29 on dist-test-slave-w4v5"
server_key: "bea9e3f51e111ecec871ed0ed804f814"
server_key_iv: "53bbff95c620cf147ea6491c0489ab56"
server_key_version: "encryptionkey@0"
I20260812 16:37:29.390532 11459 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:37:29.391396 11484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:29.391564 11459 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:37:29.391702 11459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-0/wal
uuid: "1150defc97664bd989f0eee19b7140aa"
format_stamp: "Formatted at 2026-08-12 16:37:29 on dist-test-slave-w4v5"
server_key: "bea9e3f51e111ecec871ed0ed804f814"
server_key_iv: "53bbff95c620cf147ea6491c0489ab56"
server_key_version: "encryptionkey@0"
I20260812 16:37:29.391786 11459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:29.406388 11459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:29.408526 11459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:29.408694 11459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:29.408936 11459 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:37:29.409403 11459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:29.409475 11459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:29.409515 11459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:29.409543 11459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:29.415827 11459 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.193:39885
I20260812 16:37:29.415876 11681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.193:39885 every 8 connection(s)
I20260812 16:37:29.416195 11459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
Aug 12 16:37:29 dist-test-slave-w4v5 krb5kdc[11267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552649, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:29.423869 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 11459
I20260812 16:37:29.423991 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance
I20260812 16:37:29.424220 1015 external_mini_cluster.cc:1428] Setting key 9483c9df343b34e4e25bc724f22ed23e
I20260812 16:37:29.426120 11414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:29.416890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:40793 (local address 127.0.253.254:34093)
0812 16:37:29.416957 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:29.416960 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:29.417158 (+ 198us) server_negotiation.cc:407] Connection header received
0812 16:37:29.418229 (+ 1071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:29.418231 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:29.418244 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:29.418296 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:29.418960 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:29.419546 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:29.420543 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:29.420778 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:29.420855 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:29.423040 (+ 2185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:29.423042 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:29.423042 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:29.423048 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:29.424708 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:29.425156 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:29.425157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:29.425158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:29.425221 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:29.425475 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:29.425476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:29.425476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:29.425592 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:29.425667 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:29.425935 (+ 268us) server_negotiation.cc:299] Negotiation successful
0812 16:37:29.426030 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 16:37:29.426321 11687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:29.416866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:34093 (local address 127.0.253.193:40793)
0812 16:37:29.417023 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:29.417060 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:29.418074 (+ 1014us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:29.418373 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:29.418377 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:29.418637 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:29.418858 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:29.418863 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:29.419641 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:29.419643 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:29.420451 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:29.420452 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:29.420862 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:29.420864 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:29.420994 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:29.421529 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:29.421538 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:29.422947 (+ 1409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:29.424786 (+ 1839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:29.424794 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:29.424797 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:29.425081 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:29.425290 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:29.425291 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:29.425291 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:29.425405 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:29.425663 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:29.425671 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:29.425808 (+ 137us) client_negotiation.cc:769] Sending connection context
0812 16:37:29.425912 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 16:37:29.426065 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":69,"threads_started":1}
I20260812 16:37:29.427004 11684 heartbeater.cc:347] Connected to a master server at 127.0.253.254:34093
I20260812 16:37:29.427147 11684 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:29.427323 11684 heartbeater.cc:511] Master 127.0.253.254:34093 requested a full tablet report, sending...
I20260812 16:37:29.427860 11346 ts_manager.cc:194] Registered new tserver with Master: 1150defc97664bd989f0eee19b7140aa (127.0.253.193:39885)
I20260812 16:37:29.428458 11346 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.193@KRBTEST.COM'} at 127.0.253.193:40793
WARNING: no policy specified for kudu/127.0.253.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab.
Entry for principal kudu/127.0.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab.
I20260812 16:37:29.450691 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.194:0
--local_ip_for_outbound_sockets=127.0.253.194
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:34093
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:36569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:29.534983 11692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:29.535250 11692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:29.535277 11692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:29.536546 11692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:29.536598 11692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:29.536625 11692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.194
I20260812 16:37:29.538189 11692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:36569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:34093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.194
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:29.538455 11692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:29.538702 11692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:29.543919 11703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:29.543927 11700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:29.544062 11705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:29.544298 11692 server_base.cc:1034] running on GCE node
I20260812 16:37:29.544514 11692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:29.544716 11692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:29.545859 11692 hybrid_clock.cc:648] HybridClock initialized: now 1786552649545845 us; error 30 us; skew 500 ppm
Aug 12 16:37:29 dist-test-slave-w4v5 krb5kdc[11267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552649, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:29.548524 11692 init.cc:377] Logged in from keytab as kudu/127.0.253.194@KRBTEST.COM (short username kudu)
I20260812 16:37:29.548928 11692 webserver.cc:460] Webserver started at http://127.0.253.194:38697/ using document root <none> and password file <none>
I20260812 16:37:29.549111 11692 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:29.549152 11692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:29.549249 11692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:29.550283 11692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-1/data/instance:
uuid: "4c2b14fca57748aca7bc91f8def1870e"
format_stamp: "Formatted at 2026-08-12 16:37:29 on dist-test-slave-w4v5"
server_key: "8d247dca0c8dcdd004d2649ad34a1e23"
server_key_iv: "95a05a70217e3fd4e28de620deb676e0"
server_key_version: "encryptionkey@0"
I20260812 16:37:29.550612 11692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance:
uuid: "4c2b14fca57748aca7bc91f8def1870e"
format_stamp: "Formatted at 2026-08-12 16:37:29 on dist-test-slave-w4v5"
server_key: "8d247dca0c8dcdd004d2649ad34a1e23"
server_key_iv: "95a05a70217e3fd4e28de620deb676e0"
server_key_version: "encryptionkey@0"
I20260812 16:37:29.551983 11692 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 16:37:29.552790 11713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:29.552982 11692 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:29.553081 11692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-1/wal
uuid: "4c2b14fca57748aca7bc91f8def1870e"
format_stamp: "Formatted at 2026-08-12 16:37:29 on dist-test-slave-w4v5"
server_key: "8d247dca0c8dcdd004d2649ad34a1e23"
server_key_iv: "95a05a70217e3fd4e28de620deb676e0"
server_key_version: "encryptionkey@0"
I20260812 16:37:29.553151 11692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:29.573437 11692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:29.575902 11692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:29.576077 11692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:29.576344 11692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:37:29.576663 11692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:29.576714 11692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:29.576748 11692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:29.576767 11692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:29.582680 11692 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.194:35925
I20260812 16:37:29.582736 11894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.194:35925 every 8 connection(s)
I20260812 16:37:29.583118 11692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
I20260812 16:37:29.586207 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 11692
I20260812 16:37:29.586331 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance
I20260812 16:37:29.586519 1015 external_mini_cluster.cc:1428] Setting key a70e57e026a7e7fa2ef84eb0f9603409
Aug 12 16:37:29 dist-test-slave-w4v5 krb5kdc[11267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552649, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:29.593528 11414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:29.583894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:52047 (local address 127.0.253.254:34093)
0812 16:37:29.583950 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:29.583952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:29.584147 (+ 195us) server_negotiation.cc:407] Connection header received
0812 16:37:29.585184 (+ 1037us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:29.585185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:29.585196 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:29.585243 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:29.585884 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:29.586501 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:29.587820 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:29.588148 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:29.588227 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:29.590518 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:29.590520 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:29.590520 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:29.590527 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:29.591938 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:29.592473 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:29.592475 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:29.592475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:29.592527 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:29.592824 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:29.592825 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:29.592826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:29.592937 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:29.592993 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:29.593394 (+ 401us) server_negotiation.cc:299] Negotiation successful
0812 16:37:29.593461 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 16:37:29.593847 11898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:29.583867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:34093 (local address 127.0.253.194:52047)
0812 16:37:29.584030 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:29.584074 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:29.585019 (+ 945us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:29.585326 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:29.585330 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:29.585594 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:29.585800 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:29.585805 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:29.586598 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:29.586599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:29.587715 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:29.587719 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:29.588242 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:29.588243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:29.588356 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:29.588900 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:29.588909 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:29.590390 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:29.592025 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:29.592034 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:29.592037 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:29.592388 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:29.592591 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:29.592592 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:29.592592 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:29.592755 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:29.593008 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:29.593016 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:29.593203 (+ 187us) client_negotiation.cc:769] Sending connection context
0812 16:37:29.593335 (+ 132us) client_negotiation.cc:240] Negotiation successful
0812 16:37:29.593532 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":100,"threads_started":1}
I20260812 16:37:29.594456 11896 heartbeater.cc:347] Connected to a master server at 127.0.253.254:34093
I20260812 16:37:29.594569 11896 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:29.594763 11896 heartbeater.cc:511] Master 127.0.253.254:34093 requested a full tablet report, sending...
I20260812 16:37:29.595225 11346 ts_manager.cc:194] Registered new tserver with Master: 4c2b14fca57748aca7bc91f8def1870e (127.0.253.194:35925)
I20260812 16:37:29.595844 11346 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.194@KRBTEST.COM'} at 127.0.253.194:52047
WARNING: no policy specified for kudu/127.0.253.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab.
Entry for principal kudu/127.0.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab.
I20260812 16:37:29.615775 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.195:0
--local_ip_for_outbound_sockets=127.0.253.195
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:34093
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:36569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:29.699481 11910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:29.699760 11910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:29.699803 11910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:29.700996 11910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:29.701066 11910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:29.701116 11910 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.195
I20260812 16:37:29.702631 11910 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:36569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:34093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.195
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:29.702888 11910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:29.703168 11910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:29.708400 11919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:29.708416 11920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:29.708454 11925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:29.708678 11910 server_base.cc:1034] running on GCE node
I20260812 16:37:29.708875 11910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:29.709090 11910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:29.710238 11910 hybrid_clock.cc:648] HybridClock initialized: now 1786552649710221 us; error 31 us; skew 500 ppm
Aug 12 16:37:29 dist-test-slave-w4v5 krb5kdc[11267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552649, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:29.713200 11910 init.cc:377] Logged in from keytab as kudu/127.0.253.195@KRBTEST.COM (short username kudu)
I20260812 16:37:29.713748 11910 webserver.cc:460] Webserver started at http://127.0.253.195:44569/ using document root <none> and password file <none>
I20260812 16:37:29.713992 11910 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:29.714077 11910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:29.714229 11910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:29.715369 11910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-2/data/instance:
uuid: "545ad9fc9868488a9799025de9a1d426"
format_stamp: "Formatted at 2026-08-12 16:37:29 on dist-test-slave-w4v5"
server_key: "1809c740909f6969793c1be83166ebbc"
server_key_iv: "8100b1e51afab91c6c43f616a40d48d8"
server_key_version: "encryptionkey@0"
I20260812 16:37:29.715747 11910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance:
uuid: "545ad9fc9868488a9799025de9a1d426"
format_stamp: "Formatted at 2026-08-12 16:37:29 on dist-test-slave-w4v5"
server_key: "1809c740909f6969793c1be83166ebbc"
server_key_iv: "8100b1e51afab91c6c43f616a40d48d8"
server_key_version: "encryptionkey@0"
I20260812 16:37:29.717168 11910 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:37:29.717975 11937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:29.718170 11910 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:29.718271 11910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-2/wal
uuid: "545ad9fc9868488a9799025de9a1d426"
format_stamp: "Formatted at 2026-08-12 16:37:29 on dist-test-slave-w4v5"
server_key: "1809c740909f6969793c1be83166ebbc"
server_key_iv: "8100b1e51afab91c6c43f616a40d48d8"
server_key_version: "encryptionkey@0"
I20260812 16:37:29.718351 11910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:29.745623 11910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:29.747841 11910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:29.748013 11910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:29.748273 11910 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:37:29.748613 11910 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:29.748667 11910 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:29.748709 11910 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:29.748749 11910 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:29.754910 11910 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.195:35423
I20260812 16:37:29.754964 12140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.195:35423 every 8 connection(s)
I20260812 16:37:29.755326 11910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
I20260812 16:37:29.761829 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 11910
I20260812 16:37:29.761934 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance
Aug 12 16:37:29 dist-test-slave-w4v5 krb5kdc[11267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552649, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:29.762113 1015 external_mini_cluster.cc:1428] Setting key 3223ed6abab54343531631c21b4cc196
I20260812 16:37:29.765396 11414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:29.756114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:56645 (local address 127.0.253.254:34093)
0812 16:37:29.756167 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:29.756169 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:29.756434 (+ 265us) server_negotiation.cc:407] Connection header received
0812 16:37:29.757533 (+ 1099us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:29.757534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:29.757546 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:29.757608 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:29.758254 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:29.758748 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:29.759822 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:29.760009 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:29.760073 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:29.762510 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:29.762512 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:29.762512 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:29.762521 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:29.764067 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:29.764437 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:29.764438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:29.764438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:29.764488 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:29.764812 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:29.764813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:29.764814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:29.764944 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:29.765022 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:29.765257 (+ 235us) server_negotiation.cc:299] Negotiation successful
0812 16:37:29.765329 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 16:37:29.765666 12144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:29.756108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:34093 (local address 127.0.253.195:56645)
0812 16:37:29.756272 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:29.756309 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:29.757414 (+ 1105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:29.757672 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:29.757676 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:29.757933 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:29.758126 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:29.758131 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:29.758851 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:29.758853 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:29.759748 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:29.759751 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:29.760091 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:29.760092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:29.760193 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:29.760727 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:29.760736 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:29.762381 (+ 1645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:29.764139 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:29.764146 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:29.764148 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:29.764373 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:29.764567 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:29.764568 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:29.764568 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:29.764724 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:29.765015 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:29.765020 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:29.765144 (+ 124us) client_negotiation.cc:769] Sending connection context
0812 16:37:29.765230 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 16:37:29.765362 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":86,"threads_started":1}
I20260812 16:37:29.766145 12142 heartbeater.cc:347] Connected to a master server at 127.0.253.254:34093
I20260812 16:37:29.766239 12142 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:29.766438 12142 heartbeater.cc:511] Master 127.0.253.254:34093 requested a full tablet report, sending...
I20260812 16:37:29.766822 11346 ts_manager.cc:194] Registered new tserver with Master: 545ad9fc9868488a9799025de9a1d426 (127.0.253.195:35423)
I20260812 16:37:29.767284 11346 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.195@KRBTEST.COM'} at 127.0.253.195:56645
I20260812 16:37:29.774322 1015 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/krb5kdc/test-admin.keytab.
Aug 12 16:37:29 dist-test-slave-w4v5 krb5kdc[11267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:29.792701 1015 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260812 16:37:29.794438 12151 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:29.793632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42556 (local address 127.0.253.193:39885)
0812 16:37:29.793828 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:29.793833 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:29.793844 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:37:29.793871 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:29.793873 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:29.793892 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:29.794021 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:29.794348 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42556: BlockingRecv error: recv got EOF from 127.0.0.1:42556 (error 108)
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":103,"threads_started":1}
W20260812 16:37:29.794622 1015 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/krb5kdc/krb5cc' of type FILE
Aug 12 16:37:29 dist-test-slave-w4v5 krb5kdc[11267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:29.795745 1015 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 16:37:29 dist-test-slave-w4v5 krb5kdc[11267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:29.796710 1015 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 16:37:29 dist-test-slave-w4v5 krb5kdc[11267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.253.193@KRBTEST.COM
I20260812 16:37:29.804852 12151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:29.797213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42572 (local address 127.0.253.193:39885)
0812 16:37:29.797253 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:29.797254 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:29.797290 (+ 36us) server_negotiation.cc:407] Connection header received
0812 16:37:29.797345 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:29.797346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:29.797356 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:29.797402 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:29.797843 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:29.798620 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:29.799437 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:29.799799 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:29.799858 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:29.802071 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:29.802080 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:29.802081 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:29.802087 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:29.803683 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:29.804003 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:29.804007 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:29.804007 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:29.804070 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:29.804275 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:29.804276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:29.804277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:29.804402 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:29.804503 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:29.804635 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 16:37:29.804696 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 16:37:29.806830 12151 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:29.806163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42578 (local address 127.0.253.193:39885)
0812 16:37:29.806200 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:29.806201 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:29.806236 (+ 35us) server_negotiation.cc:407] Connection header received
0812 16:37:29.806295 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:29.806295 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:29.806305 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:29.806347 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:29.806760 (+ 413us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42578: BlockingRecv error: recv got EOF from 127.0.0.1:42578 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260812 16:37:29.806861 1015 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/krb5kdc/krb5cc' of type FILE: Runtime error: Bad format in credentials cache (filename: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/krb5kdc/krb5cc)
Aug 12 16:37:29 dist-test-slave-w4v5 krb5kdc[11267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:29.807911 1015 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 16:37:29 dist-test-slave-w4v5 krb5kdc[11267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:29.808799 1015 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 16:37:29 dist-test-slave-w4v5 krb5kdc[11267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.253.193@KRBTEST.COM
I20260812 16:37:29.815891 12151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:29.809310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42584 (local address 127.0.253.193:39885)
0812 16:37:29.809364 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:29.809365 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:29.809376 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:37:29.809436 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:29.809437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:29.809445 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:29.809505 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:29.809831 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:29.810305 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:29.810958 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:29.811193 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:29.811254 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:29.813384 (+ 2130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:29.813386 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:29.813386 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:29.813394 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:29.814678 (+ 1284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:29.815040 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:29.815041 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:29.815041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:29.815159 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:29.815351 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:29.815352 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:29.815353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:29.815458 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:29.815551 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:29.815713 (+ 162us) server_negotiation.cc:299] Negotiation successful
0812 16:37:29.815796 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
W20260812 16:37:29.817754 12151 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:29.817133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42588 (local address 127.0.253.193:39885)
0812 16:37:29.817189 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:29.817191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:29.817229 (+ 38us) server_negotiation.cc:407] Connection header received
0812 16:37:29.817305 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:29.817306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:29.817316 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:29.817383 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:29.817692 (+ 309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42588: BlockingRecv error: recv got EOF from 127.0.0.1:42588 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260812 16:37:29.817924 1015 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552619305671-1015-0/krb5kdc/krb5cc)
Aug 12 16:37:29 dist-test-slave-w4v5 krb5kdc[11267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:29.818877 1015 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 16:37:29 dist-test-slave-w4v5 krb5kdc[11267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:29.819828 1015 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 16:37:29 dist-test-slave-w4v5 krb5kdc[11267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.253.193@KRBTEST.COM
I20260812 16:37:29.827174 12151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:29.820469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42602 (local address 127.0.253.193:39885)
0812 16:37:29.820559 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:29.820561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:29.820573 (+ 12us) server_negotiation.cc:407] Connection header received
0812 16:37:29.820626 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:29.820626 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:29.820634 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:29.820682 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:29.821077 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:29.821557 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:29.822317 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:29.822499 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:29.822591 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:29.824742 (+ 2151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:29.824743 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:29.824743 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:29.824749 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:29.826148 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:29.826457 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:29.826458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:29.826458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:29.826506 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:29.826734 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:29.826734 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:29.826734 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:29.826823 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:29.826894 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:29.826987 (+ 93us) server_negotiation.cc:299] Negotiation successful
0812 16:37:29.827041 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 16:37:29.827884 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 11459
I20260812 16:37:29.848610 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 11692
I20260812 16:37:29.863739 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 11910
I20260812 16:37:29.877529 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 11285
2026-08-12T16:37:29Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3664 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:37:29 dist-test-slave-w4v5 krb5kdc[12180](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:37:29 dist-test-slave-w4v5 krb5kdc[12180](info): set up 2 sockets
Aug 12 16:37:29 dist-test-slave-w4v5 krb5kdc[12180](info): commencing operation
krb5kdc: starting...
W20260812 16:37:31.195399 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.288s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:37:31 dist-test-slave-w4v5 krb5kdc[12180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:37:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:37:31Z Disabled control of system clock
WARNING: no policy specified for oryx/127.0.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.254.keytab.
Entry for principal oryx/127.0.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.254.keytab.
I20260812 16:37:31.322122 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:39023
--webserver_interface=127.0.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.254.keytab
--principal=oryx/127.0.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:40743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:39023
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:31.404827 12202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:31.405066 12202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:31.405100 12202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:31.406229 12202 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:37:31.406288 12202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:31.406313 12202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:31.406342 12202 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:37:31.406371 12202 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:37:31.407954 12202 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:40743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:39023
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.254.keytab
--principal=oryx/127.0.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:39023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:31.408228 12202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:31.408444 12202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:31.413292 12213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:31.413331 12218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:31.413354 12212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:31.413532 12202 server_base.cc:1034] running on GCE node
I20260812 16:37:31.413836 12202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:31.414084 12202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:31.415218 12202 hybrid_clock.cc:648] HybridClock initialized: now 1786552651415198 us; error 36 us; skew 500 ppm
Aug 12 16:37:31 dist-test-slave-w4v5 krb5kdc[12180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552651, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:31.420560 12202 init.cc:377] Logged in from keytab as oryx/127.0.253.254@KRBTEST.COM (short username oryx)
I20260812 16:37:31.421064 12202 webserver.cc:460] Webserver started at http://127.0.253.254:37329/ using document root <none> and password file <none>
I20260812 16:37:31.421304 12202 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:31.421353 12202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:31.421517 12202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:31.422638 12202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "283116b420234f088056a1f917a31bb7"
format_stamp: "Formatted at 2026-08-12 16:37:31 on dist-test-slave-w4v5"
server_key: "d3c7f95239dd2a40d860fd887af5d87d"
server_key_iv: "7783dbbccb5a54826502daa6a132b3a6"
server_key_version: "encryptionkey@0"
I20260812 16:37:31.423017 12202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "283116b420234f088056a1f917a31bb7"
format_stamp: "Formatted at 2026-08-12 16:37:31 on dist-test-slave-w4v5"
server_key: "d3c7f95239dd2a40d860fd887af5d87d"
server_key_iv: "7783dbbccb5a54826502daa6a132b3a6"
server_key_version: "encryptionkey@0"
I20260812 16:37:31.424412 12202 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:37:31.425195 12227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:31.425380 12202 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 16:37:31.425477 12202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "283116b420234f088056a1f917a31bb7"
format_stamp: "Formatted at 2026-08-12 16:37:31 on dist-test-slave-w4v5"
server_key: "d3c7f95239dd2a40d860fd887af5d87d"
server_key_iv: "7783dbbccb5a54826502daa6a132b3a6"
server_key_version: "encryptionkey@0"
I20260812 16:37:31.425554 12202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:31.467514 12202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:31.469521 12202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:31.469734 12202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:31.473901 12202 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:39023
I20260812 16:37:31.473938 12310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:39023 every 8 connection(s)
I20260812 16:37:31.474319 12202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:37:31.475505 12314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:31.477763 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 12202
I20260812 16:37:31.477874 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:37:31.478113 1015 external_mini_cluster.cc:1428] Setting key f9edd37813f7006af24ad7a250dff257
I20260812 16:37:31.478394 12314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7: Bootstrap starting.
I20260812 16:37:31.479202 12314 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:31.479537 12314 log.cc:824] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:31.480314 12314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7: No bootstrap required, opened a new log
I20260812 16:37:31.482666 12314 raft_consensus.cc:348] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } }
I20260812 16:37:31.482841 12314 raft_consensus.cc:374] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:31.482872 12314 raft_consensus.cc:729] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 283116b420234f088056a1f917a31bb7, State: Initialized, Role: FOLLOWER
I20260812 16:37:31.482992 12314 consensus_queue.cc:260] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } }
I20260812 16:37:31.483110 12314 raft_consensus.cc:388] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:37:31.483155 12314 raft_consensus.cc:482] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:37:31.483208 12314 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:31.483845 12314 raft_consensus.cc:504] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } }
I20260812 16:37:31.483971 12314 leader_election.cc:302] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 283116b420234f088056a1f917a31bb7; no voters:
I20260812 16:37:31.484150 12314 leader_election.cc:288] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:37:31.484217 12323 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:37:31.484411 12314 sys_catalog.cc:564] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:37:31.484431 12323 raft_consensus.cc:686] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [term 1 LEADER]: Becoming Leader. State: Replica: 283116b420234f088056a1f917a31bb7, State: Running, Role: LEADER
I20260812 16:37:31.484692 12323 consensus_queue.cc:237] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } }
I20260812 16:37:31.485061 12325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "283116b420234f088056a1f917a31bb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } }
I20260812 16:37:31.485164 12325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:31.485052 12323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 283116b420234f088056a1f917a31bb7. Latest consensus state: current_term: 1 leader_uuid: "283116b420234f088056a1f917a31bb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } }
I20260812 16:37:31.485371 12323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [sys.catalog]: This master's current role is: LEADER
Aug 12 16:37:31 dist-test-slave-w4v5 krb5kdc[12180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM
I20260812 16:37:31.486117 12334 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:37:31.486635 12334 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:37:31.488296 12334 catalog_manager.cc:1347] Generated new cluster ID: 9d5f31f6600341f5982998fd8c1db852
I20260812 16:37:31.488345 12334 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:37:31.489811 12320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:31.478710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54280 (local address 127.0.253.254:39023)
0812 16:37:31.478926 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:31.478936 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:31.478973 (+ 37us) server_negotiation.cc:407] Connection header received
0812 16:37:31.479649 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:31.479658 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:31.479728 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:31.479889 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:31.480583 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:31.481970 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:31.482686 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:31.483340 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:31.483455 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:31.486400 (+ 2945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:31.486404 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:31.486404 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:31.486415 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:31.488263 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:31.488652 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:31.488655 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:31.488656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:31.488745 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:31.488958 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:31.488959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:31.488960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:31.489199 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:31.489336 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:31.489456 (+ 120us) server_negotiation.cc:299] Negotiation successful
0812 16:37:31.489593 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":96,"threads_started":1}
I20260812 16:37:31.517485 12334 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:37:31.518093 12334 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:37:31.527431 12334 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7: Generated new TSK 0
I20260812 16:37:31.527668 12334 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.0.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.193.keytab.
Entry for principal oryx/127.0.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.193.keytab.
I20260812 16:37:31.566746 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.193:0
--local_ip_for_outbound_sockets=127.0.253.193
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:39023
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.193.keytab
--principal=oryx/127.0.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:40743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:31.652012 12354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:31.652218 12354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:31.652242 12354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:31.653396 12354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:31.653438 12354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:31.653465 12354 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.193
I20260812 16:37:31.654951 12354 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:40743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.193.keytab
--principal=oryx/127.0.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:39023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12354
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.193
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:31.655224 12354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:31.655483 12354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:31.660494 12361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:31.660516 12360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:31.660553 12365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:31.660665 12354 server_base.cc:1034] running on GCE node
I20260812 16:37:31.660918 12354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:31.661182 12354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:31.662348 12354 hybrid_clock.cc:648] HybridClock initialized: now 1786552651662325 us; error 44 us; skew 500 ppm
Aug 12 16:37:31 dist-test-slave-w4v5 krb5kdc[12180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552651, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:31.664909 12354 init.cc:377] Logged in from keytab as oryx/127.0.253.193@KRBTEST.COM (short username oryx)
I20260812 16:37:31.665349 12354 webserver.cc:460] Webserver started at http://127.0.253.193:39529/ using document root <none> and password file <none>
I20260812 16:37:31.665597 12354 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:31.665663 12354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:31.665784 12354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:31.666790 12354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-0/data/instance:
uuid: "826e312a29ed44e286fbd752c626acfd"
format_stamp: "Formatted at 2026-08-12 16:37:31 on dist-test-slave-w4v5"
server_key: "922b33a39de1077ba034028bfbaca9c9"
server_key_iv: "d3e1db71ea1bff1353d0af504e6e16d8"
server_key_version: "encryptionkey@0"
I20260812 16:37:31.667207 12354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance:
uuid: "826e312a29ed44e286fbd752c626acfd"
format_stamp: "Formatted at 2026-08-12 16:37:31 on dist-test-slave-w4v5"
server_key: "922b33a39de1077ba034028bfbaca9c9"
server_key_iv: "d3e1db71ea1bff1353d0af504e6e16d8"
server_key_version: "encryptionkey@0"
I20260812 16:37:31.668540 12354 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:31.669224 12378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:31.669389 12354 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:37:31.669492 12354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-0/wal
uuid: "826e312a29ed44e286fbd752c626acfd"
format_stamp: "Formatted at 2026-08-12 16:37:31 on dist-test-slave-w4v5"
server_key: "922b33a39de1077ba034028bfbaca9c9"
server_key_iv: "d3e1db71ea1bff1353d0af504e6e16d8"
server_key_version: "encryptionkey@0"
I20260812 16:37:31.669562 12354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:31.698959 12354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:31.701334 12354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:31.701510 12354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:31.701774 12354 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:37:31.702135 12354 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:31.702193 12354 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:31.702230 12354 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:31.702270 12354 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:31.708462 12354 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.193:36583
I20260812 16:37:31.708506 12569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.193:36583 every 8 connection(s)
I20260812 16:37:31.708866 12354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
I20260812 16:37:31.712661 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 12354
I20260812 16:37:31.712772 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance
I20260812 16:37:31.712975 1015 external_mini_cluster.cc:1428] Setting key b8011989b7cb2d518a1e28a1d18683e3
Aug 12 16:37:31 dist-test-slave-w4v5 krb5kdc[12180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552651, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.253.193@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM
I20260812 16:37:31.719707 12320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:31.709713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:33299 (local address 127.0.253.254:39023)
0812 16:37:31.709783 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:31.709785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:31.709972 (+ 187us) server_negotiation.cc:407] Connection header received
0812 16:37:31.710461 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:31.710462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:31.710473 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:31.710538 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:31.711572 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:31.712090 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:31.713428 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:31.713777 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:31.713846 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:31.716480 (+ 2634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:31.716482 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:31.716482 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:31.716490 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:31.718138 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:31.718558 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:31.718560 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:31.718560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:31.718636 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:31.718906 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:31.718907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:31.718907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:31.719118 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:31.719200 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:31.719494 (+ 294us) server_negotiation.cc:299] Negotiation successful
0812 16:37:31.719602 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 16:37:31.719769 12574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:31.709651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.253.193:33299)
0812 16:37:31.709840 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:31.709883 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:31.710368 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:31.710606 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:31.710609 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:31.711254 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:31.711463 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:31.711469 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:31.712181 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:31.712182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:31.713362 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:31.713366 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:31.713891 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:31.713892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:31.714057 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:31.714626 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:31.714638 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:31.716376 (+ 1738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:31.718239 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:31.718246 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:31.718248 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:31.718479 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:31.718727 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:31.718728 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:31.718728 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:31.718835 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:31.719205 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:31.719213 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:31.719346 (+ 133us) client_negotiation.cc:769] Sending connection context
0812 16:37:31.719445 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 16:37:31.719577 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":106,"threads_started":1}
I20260812 16:37:31.720737 12570 heartbeater.cc:347] Connected to a master server at 127.0.253.254:39023
I20260812 16:37:31.720857 12570 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:31.721087 12570 heartbeater.cc:511] Master 127.0.253.254:39023 requested a full tablet report, sending...
I20260812 16:37:31.721709 12253 ts_manager.cc:194] Registered new tserver with Master: 826e312a29ed44e286fbd752c626acfd (127.0.253.193:36583)
I20260812 16:37:31.722361 12253 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.253.193@KRBTEST.COM'} at 127.0.253.193:33299
WARNING: no policy specified for oryx/127.0.253.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.253.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.194.keytab.
Entry for principal oryx/127.0.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.194.keytab.
I20260812 16:37:31.741871 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.194:0
--local_ip_for_outbound_sockets=127.0.253.194
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:39023
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.194.keytab
--principal=oryx/127.0.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:40743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:31.828239 12579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:31.828482 12579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:31.828523 12579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:31.829792 12579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:31.829859 12579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:31.829907 12579 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.194
I20260812 16:37:31.831494 12579 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:40743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.194.keytab
--principal=oryx/127.0.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:39023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.194
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:31.831758 12579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:31.832031 12579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:31.837527 12587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:31.837554 12592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:31.837596 12586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:31.837651 12579 server_base.cc:1034] running on GCE node
I20260812 16:37:31.837970 12579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:31.838250 12579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:31.839417 12579 hybrid_clock.cc:648] HybridClock initialized: now 1786552651839404 us; error 44 us; skew 500 ppm
Aug 12 16:37:31 dist-test-slave-w4v5 krb5kdc[12180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552651, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:31.841991 12579 init.cc:377] Logged in from keytab as oryx/127.0.253.194@KRBTEST.COM (short username oryx)
I20260812 16:37:31.842470 12579 webserver.cc:460] Webserver started at http://127.0.253.194:41153/ using document root <none> and password file <none>
I20260812 16:37:31.842720 12579 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:31.842796 12579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:31.842947 12579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:31.844218 12579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-1/data/instance:
uuid: "8b99632ff7e84f99a05e3e8bd6435268"
format_stamp: "Formatted at 2026-08-12 16:37:31 on dist-test-slave-w4v5"
server_key: "2416942370e53804ce8c95b58f9c9847"
server_key_iv: "b14fc5fd89a1d1cb05eb6a44ffe0b0b8"
server_key_version: "encryptionkey@0"
I20260812 16:37:31.844621 12579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance:
uuid: "8b99632ff7e84f99a05e3e8bd6435268"
format_stamp: "Formatted at 2026-08-12 16:37:31 on dist-test-slave-w4v5"
server_key: "2416942370e53804ce8c95b58f9c9847"
server_key_iv: "b14fc5fd89a1d1cb05eb6a44ffe0b0b8"
server_key_version: "encryptionkey@0"
I20260812 16:37:31.846058 12579 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:37:31.846871 12606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:31.847059 12579 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:37:31.847220 12579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-1/wal
uuid: "8b99632ff7e84f99a05e3e8bd6435268"
format_stamp: "Formatted at 2026-08-12 16:37:31 on dist-test-slave-w4v5"
server_key: "2416942370e53804ce8c95b58f9c9847"
server_key_iv: "b14fc5fd89a1d1cb05eb6a44ffe0b0b8"
server_key_version: "encryptionkey@0"
I20260812 16:37:31.847283 12579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:31.858981 12579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:31.861107 12579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:31.861243 12579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:31.861438 12579 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:37:31.861711 12579 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:31.861742 12579 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:31.861766 12579 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:31.861781 12579 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:31.868602 12579 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.194:41387
I20260812 16:37:31.868729 12819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.194:41387 every 8 connection(s)
I20260812 16:37:31.869067 12579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
Aug 12 16:37:31 dist-test-slave-w4v5 krb5kdc[12180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552651, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.253.194@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM
I20260812 16:37:31.877718 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 12579
I20260812 16:37:31.877810 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance
I20260812 16:37:31.878016 1015 external_mini_cluster.cc:1428] Setting key 0e3cbe095acf122ee4a6bf9fa5b6b26d
I20260812 16:37:31.879809 12320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:31.869943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:35435 (local address 127.0.253.254:39023)
0812 16:37:31.870024 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:31.870026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:31.870286 (+ 260us) server_negotiation.cc:407] Connection header received
0812 16:37:31.870946 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:31.870947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:31.870958 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:31.871025 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:31.872175 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:31.872796 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:31.874011 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:31.874219 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:31.874287 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:31.876606 (+ 2319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:31.876607 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:31.876608 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:31.876615 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:31.878171 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:31.878666 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:31.878668 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:31.878668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:31.878739 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:31.879063 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:31.879063 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:31.879064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:31.879235 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:31.879291 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:31.879658 (+ 367us) server_negotiation.cc:299] Negotiation successful
0812 16:37:31.879741 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 16:37:31.880119 12823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:31.869918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.253.194:35435)
0812 16:37:31.870131 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:31.870200 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:31.870836 (+ 636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:31.871140 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:31.871144 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:31.871815 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:31.872083 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:31.872089 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:31.872881 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:31.872881 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:31.873868 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:31.873871 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:31.874302 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:31.874302 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:31.874409 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:31.874916 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:31.874925 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:31.876507 (+ 1582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:31.878260 (+ 1753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:31.878267 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:31.878269 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:31.878569 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:31.878820 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:31.878821 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:31.878821 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:31.878988 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:31.879305 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:31.879312 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:31.879493 (+ 181us) client_negotiation.cc:769] Sending connection context
0812 16:37:31.879622 (+ 129us) client_negotiation.cc:240] Negotiation successful
0812 16:37:31.879822 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":86,"threads_started":1}
I20260812 16:37:31.880646 12820 heartbeater.cc:347] Connected to a master server at 127.0.253.254:39023
I20260812 16:37:31.880765 12820 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:31.880995 12820 heartbeater.cc:511] Master 127.0.253.254:39023 requested a full tablet report, sending...
I20260812 16:37:31.881415 12253 ts_manager.cc:194] Registered new tserver with Master: 8b99632ff7e84f99a05e3e8bd6435268 (127.0.253.194:41387)
I20260812 16:37:31.881986 12253 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.253.194@KRBTEST.COM'} at 127.0.253.194:35435
WARNING: no policy specified for oryx/127.0.253.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.195.keytab.
Entry for principal oryx/127.0.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.195.keytab.
I20260812 16:37:31.906196 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.195:0
--local_ip_for_outbound_sockets=127.0.253.195
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:39023
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.195.keytab
--principal=oryx/127.0.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:40743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:31.989876 12827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:31.990103 12827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:31.990154 12827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:31.991392 12827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:31.991451 12827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:31.991495 12827 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.195
I20260812 16:37:31.992956 12827 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:40743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.195.keytab
--principal=oryx/127.0.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:39023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.195
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:31.993220 12827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:31.993451 12827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:31.998703 12847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:31.998772 12843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:31.998701 12841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:31.998947 12827 server_base.cc:1034] running on GCE node
I20260812 16:37:31.999185 12827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:31.999439 12827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:32.000607 12827 hybrid_clock.cc:648] HybridClock initialized: now 1786552652000601 us; error 35 us; skew 500 ppm
Aug 12 16:37:31 dist-test-slave-w4v5 krb5kdc[12180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552651, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:32.003401 12827 init.cc:377] Logged in from keytab as oryx/127.0.253.195@KRBTEST.COM (short username oryx)
I20260812 16:37:32.003837 12827 webserver.cc:460] Webserver started at http://127.0.253.195:36813/ using document root <none> and password file <none>
I20260812 16:37:32.004061 12827 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:32.004127 12827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:32.004280 12827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:32.005379 12827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-2/data/instance:
uuid: "0af3a5dcce9d4ae890aecc4f9f4ff6f7"
format_stamp: "Formatted at 2026-08-12 16:37:32 on dist-test-slave-w4v5"
server_key: "f1c1c5e7eb44cdcee06d90b8585f7b3b"
server_key_iv: "8ab88f0ab39abb4a437c5f85846f417f"
server_key_version: "encryptionkey@0"
I20260812 16:37:32.005765 12827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance:
uuid: "0af3a5dcce9d4ae890aecc4f9f4ff6f7"
format_stamp: "Formatted at 2026-08-12 16:37:32 on dist-test-slave-w4v5"
server_key: "f1c1c5e7eb44cdcee06d90b8585f7b3b"
server_key_iv: "8ab88f0ab39abb4a437c5f85846f417f"
server_key_version: "encryptionkey@0"
I20260812 16:37:32.007261 12827 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260812 16:37:32.008042 12861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:32.008215 12827 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:37:32.008317 12827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-2/wal
uuid: "0af3a5dcce9d4ae890aecc4f9f4ff6f7"
format_stamp: "Formatted at 2026-08-12 16:37:32 on dist-test-slave-w4v5"
server_key: "f1c1c5e7eb44cdcee06d90b8585f7b3b"
server_key_iv: "8ab88f0ab39abb4a437c5f85846f417f"
server_key_version: "encryptionkey@0"
I20260812 16:37:32.008387 12827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:32.019248 12827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:32.021673 12827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:32.021847 12827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:32.022109 12827 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:37:32.022430 12827 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:32.022481 12827 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:32.022522 12827 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:32.022548 12827 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:32.028728 12827 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.195:43725
I20260812 16:37:32.028825 13068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.195:43725 every 8 connection(s)
I20260812 16:37:32.029183 12827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
I20260812 16:37:32.032166 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 12827
I20260812 16:37:32.032280 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance
I20260812 16:37:32.032536 1015 external_mini_cluster.cc:1428] Setting key dbebefcdc16ee7e4ca47ba9272755111
Aug 12 16:37:32 dist-test-slave-w4v5 krb5kdc[12180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552651, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.253.195@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM
I20260812 16:37:32.039495 12320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.030024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:35585 (local address 127.0.253.254:39023)
0812 16:37:32.030088 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:32.030090 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:32.030309 (+ 219us) server_negotiation.cc:407] Connection header received
0812 16:37:32.030907 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:32.030909 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:32.030923 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:32.030975 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:32.032192 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.033038 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.034202 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.034411 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.034494 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.036648 (+ 2154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:32.036650 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:32.036650 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:32.036657 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:32.038037 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.038451 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.038452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.038452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.038510 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.038760 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.038761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.038761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.038868 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:32.038922 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:32.039338 (+ 416us) server_negotiation.cc:299] Negotiation successful
0812 16:37:32.039429 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 16:37:32.039747 13072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.030022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.253.195:35585)
0812 16:37:32.030192 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:32.030228 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:32.030787 (+ 559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:32.031051 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:32.031054 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:32.031802 (+ 748us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:32.032071 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.032079 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.033131 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.033132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.034094 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.034097 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.034505 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.034506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.034647 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.035188 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:32.035197 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:32.036561 (+ 1364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:32.038120 (+ 1559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.038127 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.038129 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.038376 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.038574 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.038575 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.038575 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.038701 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.038954 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:32.038961 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:32.039187 (+ 226us) client_negotiation.cc:769] Sending connection context
0812 16:37:32.039301 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 16:37:32.039447 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":96,"threads_started":1}
I20260812 16:37:32.040336 13069 heartbeater.cc:347] Connected to a master server at 127.0.253.254:39023
I20260812 16:37:32.040436 13069 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:32.040592 13069 heartbeater.cc:511] Master 127.0.253.254:39023 requested a full tablet report, sending...
I20260812 16:37:32.040968 12253 ts_manager.cc:194] Registered new tserver with Master: 0af3a5dcce9d4ae890aecc4f9f4ff6f7 (127.0.253.195:43725)
I20260812 16:37:32.041361 12253 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.253.195@KRBTEST.COM'} at 127.0.253.195:35585
I20260812 16:37:32.044315 1015 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.0.253.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.253.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.253.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.253.keytab.
Entry for principal oryx/127.0.253.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.253.keytab.
I20260812 16:37:32.071247 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.253:43719
--webserver_interface=127.0.253.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.253.keytab
--principal=oryx/127.0.253.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:40743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:39023,127.0.253.253:43719
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:32.154927 13077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:32.155148 13077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:32.155172 13077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:32.156323 13077 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:37:32.156363 13077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:32.156378 13077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:32.156389 13077 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:37:32.156401 13077 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:37:32.157840 13077 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:40743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:39023,127.0.253.253:43719
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.253.keytab
--principal=oryx/127.0.253.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.253:43719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.253.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13077
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
W20260812 16:37:32.157940 13077 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.0.253.254:39023,127.0.253.253:43719'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260812 16:37:32.158048 13077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:32.158226 13077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 16:37:32.163441 13077 server_base.cc:1034] running on GCE node
W20260812 16:37:32.163424 13084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:32.163424 13090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:32.163446 13085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:32.163964 13077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:32.164230 13077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:32.165416 13077 hybrid_clock.cc:648] HybridClock initialized: now 1786552652165394 us; error 41 us; skew 500 ppm
Aug 12 16:37:32 dist-test-slave-w4v5 krb5kdc[12180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552652, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.253.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:32.168298 13077 init.cc:377] Logged in from keytab as oryx/127.0.253.253@KRBTEST.COM (short username oryx)
I20260812 16:37:32.168779 13077 webserver.cc:460] Webserver started at http://127.0.253.253:42369/ using document root <none> and password file <none>
I20260812 16:37:32.169026 13077 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:32.169092 13077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:32.169253 13077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:32.170307 13077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-1/data/instance:
uuid: "9f12b08685a648f6ad5f1cca4c111d97"
format_stamp: "Formatted at 2026-08-12 16:37:32 on dist-test-slave-w4v5"
server_key: "5e6c3cb3057cb731ed157786e2594a53"
server_key_iv: "f368e0af9c1283bf28e98f44dd119e24"
server_key_version: "encryptionkey@0"
I20260812 16:37:32.170672 13077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-1/wal/instance:
uuid: "9f12b08685a648f6ad5f1cca4c111d97"
format_stamp: "Formatted at 2026-08-12 16:37:32 on dist-test-slave-w4v5"
server_key: "5e6c3cb3057cb731ed157786e2594a53"
server_key_iv: "f368e0af9c1283bf28e98f44dd119e24"
server_key_version: "encryptionkey@0"
I20260812 16:37:32.172109 13077 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:37:32.172905 13101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:32.173065 13077 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:32.173164 13077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-1/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-1/wal
uuid: "9f12b08685a648f6ad5f1cca4c111d97"
format_stamp: "Formatted at 2026-08-12 16:37:32 on dist-test-slave-w4v5"
server_key: "5e6c3cb3057cb731ed157786e2594a53"
server_key_iv: "f368e0af9c1283bf28e98f44dd119e24"
server_key_version: "encryptionkey@0"
I20260812 16:37:32.173241 13077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:32.182981 13077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:32.184998 13077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:32.185176 13077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:32.189543 13077 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.253:43719
I20260812 16:37:32.189577 13178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.253:43719 every 8 connection(s)
I20260812 16:37:32.189934 13077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-1/data/info.pb
I20260812 16:37:32.190946 13185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:32.192539 13185 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:37:32.196835 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 13077
Aug 12 16:37:32 dist-test-slave-w4v5 krb5kdc[12180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552652, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.253.253@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM
I20260812 16:37:32.201783 12320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.193250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54286 (local address 127.0.253.254:39023)
0812 16:37:32.193335 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:32.193338 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:32.193474 (+ 136us) server_negotiation.cc:407] Connection header received
0812 16:37:32.193946 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:32.193947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:32.193958 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:32.194011 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:32.194564 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.195115 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.196318 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.196534 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.196599 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.198787 (+ 2188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:32.198789 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:32.198789 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:32.198797 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:32.200341 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.200795 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.200796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.200796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.200854 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.201113 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.201114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.201114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.201267 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:32.201344 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:32.201607 (+ 263us) server_negotiation.cc:299] Negotiation successful
0812 16:37:32.201692 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 16:37:32.201982 13188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.193188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.0.1:54286)
0812 16:37:32.193356 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:32.193393 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:32.193864 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:32.194061 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:32.194064 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:32.194309 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:32.194497 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.194501 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.195209 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.195210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.196235 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.196238 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.196643 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.196644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.196802 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.197307 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:32.197317 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:32.198696 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:32.200419 (+ 1723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.200425 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.200427 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.200709 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.200924 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.200924 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.200925 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.201039 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.201335 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:32.201341 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:32.201476 (+ 135us) client_negotiation.cc:769] Sending connection context
0812 16:37:32.201584 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 16:37:32.201743 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":96,"threads_started":1}
Aug 12 16:37:32 dist-test-slave-w4v5 krb5kdc[12180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.253.253@KRBTEST.COM
I20260812 16:37:32.202664 13185 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:37:32.205883 13191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.197304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38690 (local address 127.0.253.253:43719)
0812 16:37:32.197722 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:32.197726 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:32.197741 (+ 15us) server_negotiation.cc:407] Connection header received
0812 16:37:32.197758 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:32.197759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:32.197776 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:32.197856 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:32.198469 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.199182 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.200044 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.200523 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.200574 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.203049 (+ 2475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:32.203057 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:32.203057 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:32.203064 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:32.204728 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.205064 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.205067 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.205067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.205133 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.205331 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.205332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.205333 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.205468 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:32.205640 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:32.205694 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 16:37:32.205768 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":87,"threads_started":1}
Aug 12 16:37:32 dist-test-slave-w4v5 krb5kdc[12180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552652, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.253.253@KRBTEST.COM for oryx/127.0.253.253@KRBTEST.COM
I20260812 16:37:32.210363 13188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.202917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.253:43719 (local address 127.0.0.1:38692)
0812 16:37:32.202974 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:32.202983 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:32.203031 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:32.203250 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:32.203251 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:32.203395 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:32.203465 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.203467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.204259 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.204260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.204959 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.204960 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.205333 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.205334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.205397 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.205769 (+ 372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:32.205774 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:32.207148 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:32.209322 (+ 2174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.209323 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.209324 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.209487 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.209713 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.209714 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.209714 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.209767 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.210003 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:32.210004 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:32.210045 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 16:37:32.210140 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 16:37:32.210256 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 16:37:32.210419 13194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.202972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38692 (local address 127.0.253.253:43719)
0812 16:37:32.203130 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:32.203132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:32.203145 (+ 13us) server_negotiation.cc:407] Connection header received
0812 16:37:32.203167 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:32.203167 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:32.203175 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:32.203227 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:32.203530 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.204183 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.205044 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.205248 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.205332 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.207250 (+ 1918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:32.207251 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:32.207251 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:32.207257 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:32.209225 (+ 1968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.209599 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.209600 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.209600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.209649 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.209839 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.209840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.209840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.209940 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:32.210012 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:32.210162 (+ 150us) server_negotiation.cc:299] Negotiation successful
0812 16:37:32.210290 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":139,"thread_start_us":115,"threads_started":1}
I20260812 16:37:32.211725 13185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97: Bootstrap starting.
I20260812 16:37:32.212365 13185 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:32.212652 13185 log.cc:824] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:32.213478 13185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97: No bootstrap required, opened a new log
I20260812 16:37:32.214918 13185 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } }
I20260812 16:37:32.215042 13185 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:32.215066 13185 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f12b08685a648f6ad5f1cca4c111d97, State: Initialized, Role: FOLLOWER
I20260812 16:37:32.215322 13185 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } }
I20260812 16:37:32.215588 13197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } } }
I20260812 16:37:32.215641 13185 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:37:32.215699 13197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 16:37:32.217396 13222 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 16:37:32.217456 13222 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:37:32.219578 13077 master_runner.cc:428] Detected that this master 9f12b08685a648f6ad5f1cca4c111d97 is joining an existing cluster
I20260812 16:37:32.219655 13077 master_runner.cc:432] Initiating AddMaster RPC to add 127.0.253.253:43719
I20260812 16:37:32.228410 12320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.220882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54294 (local address 127.0.253.254:39023)
0812 16:37:32.220970 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:32.220972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:32.221244 (+ 272us) server_negotiation.cc:407] Connection header received
0812 16:37:32.221356 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:32.221358 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:32.221371 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:32.221433 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:32.221910 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.222696 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.223792 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.224052 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.224144 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.225195 (+ 1051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:32.225196 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:32.225196 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:32.225204 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:32.226885 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.227487 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.227489 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.227489 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.227551 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.227793 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.227794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.227794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.227980 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:32.228062 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:32.228227 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 16:37:32.228311 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 16:37:32.228407 13229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.220878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.253:43719 (local address 127.0.0.1:38702)
0812 16:37:32.221041 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:32.221053 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:32.221148 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:32.221304 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:32.221305 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:32.221601 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:32.221679 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.221681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.222447 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.222448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.223184 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.223187 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.223605 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.223606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.223682 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.224420 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:32.224424 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:32.224981 (+ 557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:32.226976 (+ 1995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.226977 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.226978 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.227352 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.227596 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.227597 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.227597 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.227652 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.227957 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:32.227958 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:32.228002 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 16:37:32.228103 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 16:37:32.228236 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":105,"threads_started":1}
I20260812 16:37:32.228415 13194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.220926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38702 (local address 127.0.253.253:43719)
0812 16:37:32.221001 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:32.221003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:32.221117 (+ 114us) server_negotiation.cc:407] Connection header received
0812 16:37:32.221230 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:32.221231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:32.221241 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:32.221291 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:32.221761 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.222365 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.223280 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.223509 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.223593 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.225076 (+ 1483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:32.225078 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:32.225078 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:32.225086 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:32.226877 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.227448 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.227449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.227450 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.227512 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.227705 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.227707 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.227707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.227847 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:32.227940 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:32.228147 (+ 207us) server_negotiation.cc:299] Negotiation successful
0812 16:37:32.228304 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":58}
I20260812 16:37:32.229220 13233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.221039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.0.1:54294)
0812 16:37:32.221179 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:32.221188 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:32.221271 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:32.221451 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:32.221452 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:32.221729 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:32.221809 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.221811 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.222799 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.222800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.223695 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.223698 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.224181 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.224182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.224273 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.224677 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:32.224682 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:32.225104 (+ 422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:32.227150 (+ 2046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.227151 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.227152 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.227427 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.227658 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.227659 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.227659 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.227731 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.228048 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:32.228048 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:32.228090 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 16:37:32.228622 (+ 532us) client_negotiation.cc:240] Negotiation successful
0812 16:37:32.228738 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":90,"threads_started":1}
Aug 12 16:37:32 dist-test-slave-w4v5 krb5kdc[12180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552651, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.253.254@KRBTEST.COM for oryx/127.0.253.253@KRBTEST.COM
I20260812 16:37:32.238819 13194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.231744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38716 (local address 127.0.253.253:43719)
0812 16:37:32.231788 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:32.231790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:32.231920 (+ 130us) server_negotiation.cc:407] Connection header received
0812 16:37:32.232044 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:32.232045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:32.232055 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:32.232109 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:32.232532 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.233003 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.233829 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.234027 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.234134 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.236155 (+ 2021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:32.236157 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:32.236157 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:32.236164 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:32.237575 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.237976 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.237978 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.237978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.238040 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.238252 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.238253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.238253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.238365 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:32.238438 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:32.238642 (+ 204us) server_negotiation.cc:299] Negotiation successful
0812 16:37:32.238725 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 16:37:32.238838 13234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.231683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.253:43719 (local address 127.0.0.1:38716)
0812 16:37:32.231813 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:32.231855 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:32.231948 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:32.232116 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:32.232120 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:32.232398 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:32.232447 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.232448 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.233111 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.233113 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.233724 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.233726 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.234115 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.234116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.234207 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.234673 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:32.234677 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:32.236062 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:32.237678 (+ 1616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.237686 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.237689 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.237898 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.238110 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.238111 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.238111 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.238179 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.238431 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:32.238439 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:32.238494 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 16:37:32.238600 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 16:37:32.238730 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":65,"threads_started":1}
I20260812 16:37:32.239473 12253 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.0.253.253:43719: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97"
member_type: NON_VOTER
last_known_addr {
host: "127.0.253.253"
port: 43719
}
attrs {
promote: true
}
}
dest_uuid: "283116b420234f088056a1f917a31bb7"
cas_config_opid_index: -1
I20260812 16:37:32.239903 12253 consensus_queue.cc:237] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: NON_VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: true } }
I20260812 16:37:32.240475 13235 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 9f12b08685a648f6ad5f1cca4c111d97 (127.0.253.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: NON_VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: true } } }
I20260812 16:37:32.240809 13140 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:32.241724 13235 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.0.253.253:43719
I20260812 16:37:32.241750 13140 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [term 1 FOLLOWER]: Refusing update from remote peer 283116b420234f088056a1f917a31bb7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260812 16:37:32.241814 13235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "283116b420234f088056a1f917a31bb7" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: NON_VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: true } } }
I20260812 16:37:32.241876 13235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:32.242081 13077 master.cc:537] Master@127.0.253.253:43719 shutting down...
I20260812 16:37:32.242247 13237 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: NON_VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 16:37:32.243017 12228 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260812 16:37:32.243675 12228 consensus_peers.cc:487] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 -> Peer 9f12b08685a648f6ad5f1cca4c111d97 (127.0.253.253:43719): Couldn't send request to peer 9f12b08685a648f6ad5f1cca4c111d97. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 16:37:32.245682 13077 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:37:32.245787 13077 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:32.245844 13077 tablet_replica.cc:330] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97: stopping tablet replica
I20260812 16:37:32.258893 13077 master.cc:552] Master@127.0.253.253:43719 shutdown complete.
I20260812 16:37:32.258970 13077 master_runner.cc:305] Clearing existing system tablet
I20260812 16:37:32.259181 13077 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:32.259248 13077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:32.259908 13077 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260812 16:37:32.260715 13249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:32.260845 13077 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:32.260906 13077 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:37:32.260929 13077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-1/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-1/wal
uuid: "9f12b08685a648f6ad5f1cca4c111d97"
format_stamp: "Formatted at 2026-08-12 16:37:32 on dist-test-slave-w4v5"
server_key: "5e6c3cb3057cb731ed157786e2594a53"
server_key_iv: "f368e0af9c1283bf28e98f44dd119e24"
server_key_version: "encryptionkey@0"
I20260812 16:37:32.261672 13077 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:37:32.263188 13077 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 16:37:32.263257 13077 log.cc:1196] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260812 16:37:32.263468 13077 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97: Deleting consensus metadata
I20260812 16:37:32.263600 13077 master_runner.cc:315] Copying system tablet from 127.0.253.254:39023
I20260812 16:37:32.264391 13077 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97: tablet copy: Beginning tablet copy session from remote peer at address 127.0.253.254:39023
I20260812 16:37:32.271745 12320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.264882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54300 (local address 127.0.253.254:39023)
0812 16:37:32.264934 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:32.264936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:32.265031 (+ 95us) server_negotiation.cc:407] Connection header received
0812 16:37:32.265103 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:32.265104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:32.265114 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:32.265168 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:32.265616 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.266203 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.267515 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.267726 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.267795 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.268984 (+ 1189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:32.268986 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:32.268986 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:32.268993 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:32.270436 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.270872 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.270872 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.270873 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.270930 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.271186 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.271187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.271187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.271300 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:32.271351 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:32.271587 (+ 236us) server_negotiation.cc:299] Negotiation successful
0812 16:37:32.271666 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 16:37:32.271775 13261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.264772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.0.1:54300)
0812 16:37:32.264933 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:32.264946 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:32.265038 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:32.265196 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:32.265197 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:32.265428 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:32.265505 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.265508 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.266302 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.266303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.267422 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.267425 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.267818 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.267820 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.267900 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.268420 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:32.268426 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:32.268879 (+ 453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:32.270535 (+ 1656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.270536 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.270537 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.270794 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.271012 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.271013 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.271014 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.271118 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.271368 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:32.271369 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:32.271424 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 16:37:32.271527 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 16:37:32.271652 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":94,"threads_started":1}
I20260812 16:37:32.272269 12285 tablet_copy_service.cc:140] P 283116b420234f088056a1f917a31bb7: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 9f12b08685a648f6ad5f1cca4c111d97 ({username='oryx', principal='oryx/127.0.253.253@KRBTEST.COM'} at 127.0.0.1:54300)
I20260812 16:37:32.272431 12285 tablet_copy_service.cc:161] P 283116b420234f088056a1f917a31bb7: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 9f12b08685a648f6ad5f1cca4c111d97 at {username='oryx', principal='oryx/127.0.253.253@KRBTEST.COM'} at 127.0.0.1:54300: session id = 9f12b08685a648f6ad5f1cca4c111d97-00000000000000000000000000000000
I20260812 16:37:32.273864 12285 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 16:37:32.274812 13077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 16:37:32.276031 13077 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97: tablet copy: Starting download of 0 data blocks...
I20260812 16:37:32.276226 13077 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97: tablet copy: Starting download of 1 WAL segments...
I20260812 16:37:32.277426 13077 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 16:37:32.278272 12285 tablet_copy_service.cc:342] P 283116b420234f088056a1f917a31bb7: Request end of tablet copy session 9f12b08685a648f6ad5f1cca4c111d97-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.0.253.253@KRBTEST.COM'} at 127.0.0.1:54300
I20260812 16:37:32.278339 12285 tablet_copy_service.cc:433] P 283116b420234f088056a1f917a31bb7: ending tablet copy session 9f12b08685a648f6ad5f1cca4c111d97-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 9f12b08685a648f6ad5f1cca4c111d97
I20260812 16:37:32.281024 13077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:32.282245 13263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:32.282294 13265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:32.282433 13268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:32.282622 13077 server_base.cc:1034] running on GCE node
I20260812 16:37:32.282733 13077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:32.282868 13077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:32.283989 13077 hybrid_clock.cc:648] HybridClock initialized: now 1786552652283974 us; error 35 us; skew 500 ppm
Aug 12 16:37:32 dist-test-slave-w4v5 krb5kdc[12180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552652, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.253.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:32.285439 13077 init.cc:377] Logged in from keytab as oryx/127.0.253.253@KRBTEST.COM (short username oryx)
I20260812 16:37:32.285758 13077 webserver.cc:460] Webserver started at http://127.0.253.253:39655/ using document root <none> and password file <none>
I20260812 16:37:32.285880 13077 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:32.285933 13077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:32.286574 13077 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:37:32.287179 13276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:32.287328 13077 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:32.287391 13077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-1/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-1/wal
uuid: "9f12b08685a648f6ad5f1cca4c111d97"
format_stamp: "Formatted at 2026-08-12 16:37:32 on dist-test-slave-w4v5"
server_key: "5e6c3cb3057cb731ed157786e2594a53"
server_key_iv: "f368e0af9c1283bf28e98f44dd119e24"
server_key_version: "encryptionkey@0"
I20260812 16:37:32.287453 13077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:32.294926 13077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:32.296731 13077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:32.300026 13077 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.253:43719
I20260812 16:37:32.300457 13077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-1/data/info.pb
I20260812 16:37:32.300695 13366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.253:43719 every 8 connection(s)
I20260812 16:37:32.301362 13368 sys_catalog.cc:263] Verifying existing consensus state
I20260812 16:37:32.301815 13368 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97: Bootstrap starting.
I20260812 16:37:32.304492 13368 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 16:37:32.304751 13368 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97: Bootstrap complete.
I20260812 16:37:32.305071 13368 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: NON_VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: true } }
I20260812 16:37:32.305158 13368 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9f12b08685a648f6ad5f1cca4c111d97, State: Initialized, Role: LEARNER
I20260812 16:37:32.305189 13368 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: NON_VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: true } }
I20260812 16:37:32.305400 13368 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:37:32.306721 13372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: NON_VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: true } } }
I20260812 16:37:32.306941 13395 catalog_manager.cc:1259] Loaded cluster ID: 9d5f31f6600341f5982998fd8c1db852
I20260812 16:37:32.306939 13372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [sys.catalog]: This master's current role is: LEARNER
I20260812 16:37:32.307003 13395 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97: loading cluster ID for follower catalog manager: success
I20260812 16:37:32.308151 13395 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97: acquiring CA information for follower catalog manager: success
I20260812 16:37:32.308377 13395 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 16:37:32.315547 13397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.307618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38738 (local address 127.0.253.253:43719)
0812 16:37:32.308447 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:32.308449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:32.308458 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:37:32.308478 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:32.308478 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:32.308487 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:32.308544 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:32.309251 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.309901 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.310971 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.311245 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.311302 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.312810 (+ 1508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:32.312812 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:32.312812 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:32.312819 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:32.314313 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.314736 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.314736 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.314737 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.314789 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.315009 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.315009 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.315009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.315173 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:32.315253 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:32.315413 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 16:37:32.315478 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":814,"thread_start_us":60,"threads_started":1}
WARNING: no policy specified for oryx/127.0.253.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.253.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.253.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.252.keytab.
Entry for principal oryx/127.0.253.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.252.keytab.
I20260812 16:37:32.345716 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.252:41893
--webserver_interface=127.0.253.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.252.keytab
--principal=oryx/127.0.253.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:40743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:39023,127.0.253.253:43719,127.0.253.252:41893
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:32.448189 13405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:32.448405 13405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:32.448428 13405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:32.449518 13405 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:37:32.449554 13405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:32.449568 13405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:32.449581 13405 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:37:32.449594 13405 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:37:32.451216 13405 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:40743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:39023,127.0.253.253:43719,127.0.253.252:41893
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.252.keytab
--principal=oryx/127.0.253.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.252:41893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.253.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13405
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:32.451421 13405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:32.451598 13405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:32.456553 13412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:32.456568 13417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:32.456565 13411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:32.456799 13405 server_base.cc:1034] running on GCE node
I20260812 16:37:32.457058 13405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:32.457317 13405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:32.458461 13405 hybrid_clock.cc:648] HybridClock initialized: now 1786552652458446 us; error 30 us; skew 500 ppm
Aug 12 16:37:32 dist-test-slave-w4v5 krb5kdc[12180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552652, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.253.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:32.461197 13405 init.cc:377] Logged in from keytab as oryx/127.0.253.252@KRBTEST.COM (short username oryx)
I20260812 16:37:32.461752 13405 webserver.cc:460] Webserver started at http://127.0.253.252:36253/ using document root <none> and password file <none>
I20260812 16:37:32.462019 13405 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:32.462076 13405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:32.462219 13405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:32.463611 13405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-2/data/instance:
uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e"
format_stamp: "Formatted at 2026-08-12 16:37:32 on dist-test-slave-w4v5"
server_key: "8bd2d6e89ecf977199bd2f2422a11ffe"
server_key_iv: "daa3fc29314f67a4a81412976a487aa7"
server_key_version: "encryptionkey@0"
I20260812 16:37:32.464025 13405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-2/wal/instance:
uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e"
format_stamp: "Formatted at 2026-08-12 16:37:32 on dist-test-slave-w4v5"
server_key: "8bd2d6e89ecf977199bd2f2422a11ffe"
server_key_iv: "daa3fc29314f67a4a81412976a487aa7"
server_key_version: "encryptionkey@0"
I20260812 16:37:32.465328 13405 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:37:32.466022 13430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:32.466194 13405 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:37:32.466295 13405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-2/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-2/wal
uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e"
format_stamp: "Formatted at 2026-08-12 16:37:32 on dist-test-slave-w4v5"
server_key: "8bd2d6e89ecf977199bd2f2422a11ffe"
server_key_iv: "daa3fc29314f67a4a81412976a487aa7"
server_key_version: "encryptionkey@0"
I20260812 16:37:32.466348 13405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:32.488761 13405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:32.491166 13405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:32.491294 13405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:32.495293 13405 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.252:41893
I20260812 16:37:32.495350 13519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.252:41893 every 8 connection(s)
I20260812 16:37:32.495682 13405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-2/data/info.pb
I20260812 16:37:32.496694 13522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:32.498412 13522 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:37:32.502583 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 13405
Aug 12 16:37:32 dist-test-slave-w4v5 krb5kdc[12180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552652, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.253.252@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM
Aug 12 16:37:32 dist-test-slave-w4v5 krb5kdc[12180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.253.252@KRBTEST.COM
I20260812 16:37:32.508870 13524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.499217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.0.1:54304)
0812 16:37:32.499422 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:32.499469 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:32.499955 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:32.500226 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:32.500230 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:32.500487 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:32.500681 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.500685 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.501378 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.501379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.502186 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.502188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.502596 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.502597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.502782 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.503371 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:32.503388 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:32.505326 (+ 1938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:32.507332 (+ 2006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.507339 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.507341 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.507571 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.507791 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.507792 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.507792 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.507934 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.508259 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:32.508266 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:32.508426 (+ 160us) client_negotiation.cc:769] Sending connection context
0812 16:37:32.508546 (+ 120us) client_negotiation.cc:240] Negotiation successful
0812 16:37:32.508696 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":105,"threads_started":1}
I20260812 16:37:32.508872 12320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.499295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54304 (local address 127.0.253.254:39023)
0812 16:37:32.499369 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:32.499371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:32.499554 (+ 183us) server_negotiation.cc:407] Connection header received
0812 16:37:32.500066 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:32.500068 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:32.500081 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:32.500151 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:32.500777 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.501279 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.502297 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.502490 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.502574 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.505415 (+ 2841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:32.505417 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:32.505417 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:32.505424 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:32.507183 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.507652 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.507654 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.507654 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.507715 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.508037 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.508038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.508038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.508184 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:32.508245 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:32.508632 (+ 387us) server_negotiation.cc:299] Negotiation successful
0812 16:37:32.508759 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 16:37:32.509481 13522 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:37:32.512681 13528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.503211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35056 (local address 127.0.253.252:41893)
0812 16:37:32.503352 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:32.503357 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:32.503376 (+ 19us) server_negotiation.cc:407] Connection header received
0812 16:37:32.503406 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:32.503407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:32.503424 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:32.503489 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:32.504221 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.505186 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.505992 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.506290 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.506371 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.509072 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:32.509098 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:32.509099 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:32.509106 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:32.511051 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.511597 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.511601 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.511601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.511677 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.511927 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.511928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.511928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.512182 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:32.512365 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:32.512461 (+ 96us) server_negotiation.cc:299] Negotiation successful
0812 16:37:32.512550 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":81,"threads_started":1}
Aug 12 16:37:32 dist-test-slave-w4v5 krb5kdc[12180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552652, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.253.252@KRBTEST.COM for oryx/127.0.253.253@KRBTEST.COM
I20260812 16:37:32.517076 13397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.509826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38748 (local address 127.0.253.253:43719)
0812 16:37:32.509878 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:32.509880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:32.509889 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:37:32.509916 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:32.509918 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:32.509933 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:32.509994 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:32.510326 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.511181 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.511875 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.512098 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.512173 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.514113 (+ 1940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:32.514115 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:32.514115 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:32.514124 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:32.515767 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.516194 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.516195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.516196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.516263 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.516487 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.516488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.516488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.516628 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:32.516716 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:32.516883 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 16:37:32.516976 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 16:37:32.517099 13524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.509733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.253:43719 (local address 127.0.0.1:38748)
0812 16:37:32.509786 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:32.509799 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:32.509839 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:32.510011 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:32.510011 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:32.510181 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:32.510236 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.510237 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.511285 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.511286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.511789 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.511791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.512184 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.512185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.512260 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.512675 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:32.512682 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:32.514015 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:32.515891 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.515892 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.515893 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.516094 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.516340 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.516341 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.516342 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.516411 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.516706 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:32.516707 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:32.516762 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 16:37:32.516850 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 16:37:32.516969 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 16:37:32.517717 13522 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.253.252" port: 41893 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 16:37:32 dist-test-slave-w4v5 krb5kdc[12180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552652, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.253.252@KRBTEST.COM for oryx/127.0.253.252@KRBTEST.COM
I20260812 16:37:32.524156 13524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.518036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:41893 (local address 127.0.0.1:35060)
0812 16:37:32.518111 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:32.518123 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:32.518169 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:32.518292 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:32.518292 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:32.518439 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:32.518469 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.518470 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.519147 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.519149 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.519620 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.519621 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.520053 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.520054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.520102 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.520528 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:32.520534 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:32.521759 (+ 1225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:32.523142 (+ 1383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.523143 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.523143 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.523313 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.523515 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.523516 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.523516 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.523574 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.523816 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:32.523816 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:32.523854 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 16:37:32.523940 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 16:37:32.524041 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 16:37:32.524180 13528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.518109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35060 (local address 127.0.253.252:41893)
0812 16:37:32.518165 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:32.518167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:32.518182 (+ 15us) server_negotiation.cc:407] Connection header received
0812 16:37:32.518223 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:32.518224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:32.518235 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:32.518288 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:32.518545 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.519033 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.519758 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.519961 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.520046 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.521847 (+ 1801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:32.521848 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:32.521848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:32.521856 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:32.523016 (+ 1160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.523389 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.523390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.523390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.523448 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.523637 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.523638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.523639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.523749 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:32.523824 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:32.523964 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 16:37:32.524093 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":39}
I20260812 16:37:32.525667 13522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e: Bootstrap starting.
I20260812 16:37:32.526593 13522 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:32.526892 13522 log.cc:824] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:32.527807 13522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e: No bootstrap required, opened a new log
I20260812 16:37:32.529417 13522 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } } peers { permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 41893 } }
I20260812 16:37:32.529544 13522 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:32.529567 13522 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34e21dc2c7b040dcb9c0ad37f96e3b6e, State: Initialized, Role: FOLLOWER
I20260812 16:37:32.529757 13522 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } } peers { permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 41893 } }
I20260812 16:37:32.530077 13522 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:37:32.530102 13533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } } peers { permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 41893 } } }
I20260812 16:37:32.530212 13533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [sys.catalog]: This master's current role is: FOLLOWER
W20260812 16:37:32.531872 13557 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 16:37:32.531951 13557 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:37:32.534724 13405 master_runner.cc:428] Detected that this master 34e21dc2c7b040dcb9c0ad37f96e3b6e is joining an existing cluster
I20260812 16:37:32.534770 13405 master_runner.cc:432] Initiating AddMaster RPC to add 127.0.253.252:41893
I20260812 16:37:32.543195 13397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.536114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38750 (local address 127.0.253.253:43719)
0812 16:37:32.536173 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:32.536175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:32.536224 (+ 49us) server_negotiation.cc:407] Connection header received
0812 16:37:32.536303 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:32.536304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:32.536317 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:32.536373 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:32.536891 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.537714 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.538913 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.539273 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.539315 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.540436 (+ 1121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:32.540438 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:32.540438 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:32.540445 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:32.541847 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.542210 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.542211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.542212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.542270 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.542477 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.542478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.542479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.542596 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:32.542695 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:32.542988 (+ 293us) server_negotiation.cc:299] Negotiation successful
0812 16:37:32.543070 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 16:37:32.543218 13574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.535907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.253:43719 (local address 127.0.0.1:38750)
0812 16:37:32.536164 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:32.536175 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:32.536241 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:32.536430 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:32.536431 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:32.536713 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:32.536794 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.536796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.537806 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.537807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.538806 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.538810 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.539384 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.539385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.539440 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.539969 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:32.539975 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:32.540323 (+ 348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:32.541932 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.541933 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.541934 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.542122 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.542343 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.542344 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.542344 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.542404 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.542688 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:32.542689 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:32.542835 (+ 146us) client_negotiation.cc:769] Sending connection context
0812 16:37:32.542953 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 16:37:32.543070 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"mutex_wait_us":40,"thread_start_us":85,"threads_started":1}
I20260812 16:37:32.559003 13572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.535973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:41893 (local address 127.0.0.1:35070)
0812 16:37:32.536344 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:32.536353 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:32.536402 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:32.536569 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:32.536570 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:32.536860 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:32.536928 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.536930 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.537810 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.537811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.538815 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.538818 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.539348 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.539349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.539407 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.555533 (+ 16126us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:32.555539 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:32.556033 (+ 494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:32.557826 (+ 1793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.557827 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.557827 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.558007 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.558257 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.558258 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.558258 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.558325 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.558642 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:32.558643 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:32.558683 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 16:37:32.558752 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 16:37:32.558872 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"mutex_wait_us":61,"thread_start_us":78,"threads_started":1}
I20260812 16:37:32.559010 13528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.536124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35070 (local address 127.0.253.252:41893)
0812 16:37:32.536166 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:32.536168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:32.536443 (+ 275us) server_negotiation.cc:407] Connection header received
0812 16:37:32.536468 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:32.536469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:32.536479 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:32.536529 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:32.537016 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.537723 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.538911 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.539219 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.539261 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.556146 (+ 16885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:32.556148 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:32.556149 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:32.556156 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:32.557715 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.558121 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.558122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.558123 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.558185 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.558404 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.558406 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.558406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.558557 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:32.558622 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:32.558788 (+ 166us) server_negotiation.cc:299] Negotiation successful
0812 16:37:32.558924 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":70,"server-negotiator.queue_time_us":21}
I20260812 16:37:32.559466 12320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.535936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54312 (local address 127.0.253.254:39023)
0812 16:37:32.535987 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:32.535989 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:32.536537 (+ 548us) server_negotiation.cc:407] Connection header received
0812 16:37:32.536630 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:32.536631 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:32.536643 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:32.536686 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:32.537121 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.537847 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.538918 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.539234 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.539275 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.556272 (+ 16997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:32.556274 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:32.556274 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:32.556285 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:32.557978 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.558362 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.558363 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.558363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.558424 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.558619 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.558620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.558620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.558762 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:32.558829 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:32.559041 (+ 212us) server_negotiation.cc:299] Negotiation successful
0812 16:37:32.559369 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 16:37:32.559912 13573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.536043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.0.1:54312)
0812 16:37:32.536481 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:32.536489 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:32.536560 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:32.536755 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:32.536755 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:32.536975 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:32.537031 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.537033 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.537936 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.537936 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.538830 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.538832 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.539359 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.539359 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.539440 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.555726 (+ 16286us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:32.555731 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:32.556153 (+ 422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:32.558077 (+ 1924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.558079 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.558079 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.558279 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.558493 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.558493 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.558493 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.558546 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.558859 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:32.558860 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:32.558916 (+ 56us) client_negotiation.cc:769] Sending connection context
0812 16:37:32.559684 (+ 768us) client_negotiation.cc:240] Negotiation successful
0812 16:37:32.559798 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"mutex_wait_us":95,"thread_start_us":61,"threads_started":1}
I20260812 16:37:32.564141 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 12354
Aug 12 16:37:32 dist-test-slave-w4v5 krb5kdc[12180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552651, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.253.254@KRBTEST.COM for oryx/127.0.253.252@KRBTEST.COM
I20260812 16:37:32.569387 13234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.562025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:41893 (local address 127.0.0.1:35086)
0812 16:37:32.562089 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:32.562100 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:32.562146 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:32.562291 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:32.562292 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:32.562449 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:32.562491 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.562493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.563194 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.563195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.563808 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.563811 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.564328 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.564330 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.564407 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.564837 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:32.564842 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:32.566351 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:32.568255 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.568256 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.568256 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.568466 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.568661 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.568662 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.568662 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.568721 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.568990 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:32.568991 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:32.569044 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 16:37:32.569106 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 16:37:32.569213 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 16:37:32.569682 13528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.562091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35086 (local address 127.0.253.252:41893)
0812 16:37:32.562136 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:32.562138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:32.562146 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:37:32.562220 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:32.562221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:32.562230 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:32.562285 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:32.562567 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.563125 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.563890 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.564199 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.564271 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.566448 (+ 2177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:32.566449 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:32.566449 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:32.566456 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:32.568111 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.568541 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.568542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.568542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.568596 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.568786 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.568786 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.568787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.568910 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:32.568965 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:32.569514 (+ 549us) server_negotiation.cc:299] Negotiation successful
0812 16:37:32.569596 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 16:37:32.597577 12253 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.0.253.252:41893: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e"
member_type: NON_VOTER
last_known_addr {
host: "127.0.253.252"
port: 41893
}
attrs {
promote: true
}
}
dest_uuid: "283116b420234f088056a1f917a31bb7"
cas_config_opid_index: 5
I20260812 16:37:32.597896 12253 consensus_queue.cc:237] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: NON_VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: true } } peers { permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: NON_VOTER last_known_addr { host: "127.0.253.252" port: 41893 } attrs { promote: true } }
I20260812 16:37:32.598410 13235 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 34e21dc2c7b040dcb9c0ad37f96e3b6e (127.0.253.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: NON_VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: true } } peers { permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: NON_VOTER last_known_addr { host: "127.0.253.252" port: 41893 } attrs { promote: true } } }
I20260812 16:37:32.598658 13480 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:32.599464 13235 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.0.253.252:41893
I20260812 16:37:32.599541 13235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "283116b420234f088056a1f917a31bb7" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: NON_VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: true } } peers { permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: NON_VOTER last_known_addr { host: "127.0.253.252" port: 41893 } attrs { promote: true } } }
I20260812 16:37:32.599604 13235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:32.599956 13480 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [term 1 FOLLOWER]: Refusing update from remote peer 283116b420234f088056a1f917a31bb7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 16:37:32.600112 13405 master.cc:537] Master@127.0.253.252:41893 shutting down...
I20260812 16:37:32.600230 13237 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: NON_VOTER last_known_addr { host: "127.0.253.252" port: 41893 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 16:37:32.601047 12230 consensus_peers.cc:487] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 -> Peer 34e21dc2c7b040dcb9c0ad37f96e3b6e (127.0.253.252:41893): Couldn't send request to peer 34e21dc2c7b040dcb9c0ad37f96e3b6e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 16:37:32.603417 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.193:36583
--local_ip_for_outbound_sockets=127.0.253.193
--tserver_master_addrs=127.0.253.254:39023
--webserver_port=39529
--webserver_interface=127.0.253.193
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.193.keytab
--principal=oryx/127.0.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:40743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.0.253.254:39023,127.0.253.253:43719
--tserver_master_addrs=127.0.253.254:39023,127.0.253.253:43719,127.0.253.252:41893 with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:37:32.604075 13405 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:37:32.604135 13234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.599267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.253:43719 (local address 127.0.0.1:38758)
0812 16:37:32.599849 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:32.599861 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:32.599898 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:32.600404 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:32.600405 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:32.600407 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:32.600451 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.600452 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.601375 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.601377 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.602881 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.602884 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.603838 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.603839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.603912 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.603933 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 16:37:32.603984 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 16:37:32.604032 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563}
I20260812 16:37:32.604169 13405 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:32.604213 13405 tablet_replica.cc:330] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e: stopping tablet replica
I20260812 16:37:32.604405 13397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.600123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38758 (local address 127.0.253.253:43719)
0812 16:37:32.600172 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:32.600175 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:32.600184 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:37:32.600223 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:32.600224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:32.600249 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:32.600315 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:32.600535 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.601290 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.602971 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.603636 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.603718 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.603800 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:32.604269 (+ 469us) server_negotiation.cc:299] Negotiation successful
0812 16:37:32.604318 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 16:37:32.604604 13322 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [term 1 LEARNER]: Refusing update from remote peer 283116b420234f088056a1f917a31bb7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 16:37:32.604806 13237 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: NON_VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 16:37:32.605425 13322 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 34e21dc2c7b040dcb9c0ad37f96e3b6e (127.0.253.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: NON_VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: true } } peers { permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: NON_VOTER last_known_addr { host: "127.0.253.252" port: 41893 } attrs { promote: true } } }
I20260812 16:37:32.606868 13372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "283116b420234f088056a1f917a31bb7" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: NON_VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: true } } peers { permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: NON_VOTER last_known_addr { host: "127.0.253.252" port: 41893 } attrs { promote: true } } }
I20260812 16:37:32.607010 13372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [sys.catalog]: This master's current role is: LEARNER
I20260812 16:37:32.606851 13578 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 283116b420234f088056a1f917a31bb7. Latest consensus state: current_term: 1 leader_uuid: "283116b420234f088056a1f917a31bb7" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: NON_VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: true } } peers { permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: NON_VOTER last_known_addr { host: "127.0.253.252" port: 41893 } attrs { promote: true } } }
I20260812 16:37:32.607162 13578 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [sys.catalog]: This master's current role is: LEARNER
I20260812 16:37:32.607826 13237 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7: attempting to promote NON_VOTER 9f12b08685a648f6ad5f1cca4c111d97 to VOTER
I20260812 16:37:32.608053 13237 consensus_queue.cc:237] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: false } } peers { permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: NON_VOTER last_known_addr { host: "127.0.253.252" port: 41893 } attrs { promote: true } }
I20260812 16:37:32.608629 13234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.608502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:41893 (local address 127.0.0.1:35098)
0812 16:37:32.608551 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:32.608600 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.252:41893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 16:37:32.608636 13322 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [term 1 LEARNER]: Refusing update from remote peer 283116b420234f088056a1f917a31bb7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260812 16:37:32.608819 13235 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 16:37:32.608927 13234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.608860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:41893 (local address 127.0.0.1:35106)
0812 16:37:32.608889 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:32.608908 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.252:41893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260812 16:37:32.609009 12230 consensus_peers.cc:487] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 -> Peer 34e21dc2c7b040dcb9c0ad37f96e3b6e (127.0.253.252:41893): Couldn't send request to peer 34e21dc2c7b040dcb9c0ad37f96e3b6e. Status: Network error: Client connection negotiation failed: client connection to 127.0.253.252:41893: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 16:37:32.609433 13235 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 9f12b08685a648f6ad5f1cca4c111d97 (127.0.253.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: false } } peers { permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: NON_VOTER last_known_addr { host: "127.0.253.252" port: 41893 } attrs { promote: true } } }
I20260812 16:37:32.610563 13235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "283116b420234f088056a1f917a31bb7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: false } } peers { permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: NON_VOTER last_known_addr { host: "127.0.253.252" port: 41893 } attrs { promote: true } } }
I20260812 16:37:32.610663 13235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:32.610857 13237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "283116b420234f088056a1f917a31bb7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: false } } peers { permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: NON_VOTER last_known_addr { host: "127.0.253.252" port: 41893 } attrs { promote: true } } }
I20260812 16:37:32.610934 13237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:32.611302 13322 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 9f12b08685a648f6ad5f1cca4c111d97 (127.0.253.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: false } } peers { permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: NON_VOTER last_known_addr { host: "127.0.253.252" port: 41893 } attrs { promote: true } } }
I20260812 16:37:32.612468 13578 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "283116b420234f088056a1f917a31bb7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: false } } peers { permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: NON_VOTER last_known_addr { host: "127.0.253.252" port: 41893 } attrs { promote: true } } }
I20260812 16:37:32.612593 13578 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:37:32.618232 13405 master.cc:552] Master@127.0.253.252:41893 shutdown complete.
I20260812 16:37:32.618300 13405 master_runner.cc:305] Clearing existing system tablet
I20260812 16:37:32.618446 13405 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:32.618501 13405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:32.619294 13405 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:37:32.619880 13584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:32.620034 13405 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:32.620090 13405 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 16:37:32.620111 13405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-2/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-2/wal
uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e"
format_stamp: "Formatted at 2026-08-12 16:37:32 on dist-test-slave-w4v5"
server_key: "8bd2d6e89ecf977199bd2f2422a11ffe"
server_key_iv: "daa3fc29314f67a4a81412976a487aa7"
server_key_version: "encryptionkey@0"
I20260812 16:37:32.620851 13405 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:37:32.622255 13405 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 16:37:32.622336 13405 log.cc:1196] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260812 16:37:32.622584 13405 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e: Deleting consensus metadata
I20260812 16:37:32.622745 13405 master_runner.cc:315] Copying system tablet from 127.0.253.254:39023
I20260812 16:37:32.623418 13405 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.253.254:39023
I20260812 16:37:32.631098 12320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.623791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54314 (local address 127.0.253.254:39023)
0812 16:37:32.623840 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:32.623842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:32.623983 (+ 141us) server_negotiation.cc:407] Connection header received
0812 16:37:32.624023 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:32.624024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:32.624036 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:32.624113 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:32.624456 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.625226 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.626320 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.626609 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.626689 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.627863 (+ 1174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:32.627864 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:32.627865 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:32.627873 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:32.629645 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.630061 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.630063 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.630063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.630138 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.630377 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.630378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.630378 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.630527 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:32.630576 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:32.630864 (+ 288us) server_negotiation.cc:299] Negotiation successful
0812 16:37:32.630980 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 16:37:32.631630 13600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.623730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.0.1:54314)
0812 16:37:32.623891 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:32.623903 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:32.623975 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:32.624119 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:32.624120 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:32.624326 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:32.624381 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.624383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.625324 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.625326 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.626221 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.626223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.626701 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.626702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.626764 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.627328 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:32.627333 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:32.627769 (+ 436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:32.629750 (+ 1981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.629752 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.629752 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.629977 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.630229 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.630230 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.630230 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.630299 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.630630 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:32.630631 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:32.630699 (+ 68us) client_negotiation.cc:769] Sending connection context
0812 16:37:32.631403 (+ 704us) client_negotiation.cc:240] Negotiation successful
0812 16:37:32.631524 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":67,"threads_started":1}
I20260812 16:37:32.632084 12285 tablet_copy_service.cc:140] P 283116b420234f088056a1f917a31bb7: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 34e21dc2c7b040dcb9c0ad37f96e3b6e ({username='oryx', principal='oryx/127.0.253.252@KRBTEST.COM'} at 127.0.0.1:54314)
I20260812 16:37:32.632154 12285 tablet_copy_service.cc:161] P 283116b420234f088056a1f917a31bb7: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 34e21dc2c7b040dcb9c0ad37f96e3b6e at {username='oryx', principal='oryx/127.0.253.252@KRBTEST.COM'} at 127.0.0.1:54314: session id = 34e21dc2c7b040dcb9c0ad37f96e3b6e-00000000000000000000000000000000
I20260812 16:37:32.632566 12285 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 16:37:32.633052 13405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 16:37:32.634271 13405 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e: tablet copy: Starting download of 0 data blocks...
I20260812 16:37:32.634425 13405 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e: tablet copy: Starting download of 1 WAL segments...
I20260812 16:37:32.635315 13405 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 16:37:32.636315 12285 tablet_copy_service.cc:342] P 283116b420234f088056a1f917a31bb7: Request end of tablet copy session 34e21dc2c7b040dcb9c0ad37f96e3b6e-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.0.253.252@KRBTEST.COM'} at 127.0.0.1:54314
I20260812 16:37:32.636379 12285 tablet_copy_service.cc:433] P 283116b420234f088056a1f917a31bb7: ending tablet copy session 34e21dc2c7b040dcb9c0ad37f96e3b6e-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 34e21dc2c7b040dcb9c0ad37f96e3b6e
I20260812 16:37:32.638937 13405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:32.640120 13602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:32.644088 13604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:32.644399 13405 server_base.cc:1034] running on GCE node
W20260812 16:37:32.644502 13608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:32.644690 13405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:32.644843 13405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:32.645980 13405 hybrid_clock.cc:648] HybridClock initialized: now 1786552652645974 us; error 59 us; skew 500 ppm
Aug 12 16:37:32 dist-test-slave-w4v5 krb5kdc[12180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552652, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.253.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:32.647581 13405 init.cc:377] Logged in from keytab as oryx/127.0.253.252@KRBTEST.COM (short username oryx)
I20260812 16:37:32.647855 13405 webserver.cc:460] Webserver started at http://127.0.253.252:41537/ using document root <none> and password file <none>
I20260812 16:37:32.647982 13405 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:32.648021 13405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:32.648795 13405 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:37:32.649523 13623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:32.649674 13405 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:37:32.649727 13405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-2/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-2/wal
uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e"
format_stamp: "Formatted at 2026-08-12 16:37:32 on dist-test-slave-w4v5"
server_key: "8bd2d6e89ecf977199bd2f2422a11ffe"
server_key_iv: "daa3fc29314f67a4a81412976a487aa7"
server_key_version: "encryptionkey@0"
I20260812 16:37:32.649784 13405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:32.665126 13405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:32.667030 13405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:32.670552 13405 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.252:41893
I20260812 16:37:32.670674 13725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.252:41893 every 8 connection(s)
I20260812 16:37:32.670917 13405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/master-2/data/info.pb
I20260812 16:37:32.671851 13726 sys_catalog.cc:263] Verifying existing consensus state
I20260812 16:37:32.672372 13726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e: Bootstrap starting.
I20260812 16:37:32.675487 13726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 16:37:32.675729 13726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e: Bootstrap complete.
I20260812 16:37:32.675943 13726 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: false } } peers { permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: NON_VOTER last_known_addr { host: "127.0.253.252" port: 41893 } attrs { promote: true } }
I20260812 16:37:32.676031 13726 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 34e21dc2c7b040dcb9c0ad37f96e3b6e, State: Initialized, Role: LEARNER
I20260812 16:37:32.676066 13726 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: false } } peers { permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: NON_VOTER last_known_addr { host: "127.0.253.252" port: 41893 } attrs { promote: true } }
I20260812 16:37:32.676251 13726 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:37:32.678721 13758 catalog_manager.cc:1259] Loaded cluster ID: 9d5f31f6600341f5982998fd8c1db852
I20260812 16:37:32.678817 13758 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e: loading cluster ID for follower catalog manager: success
I20260812 16:37:32.679175 13731 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: false } } peers { permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: NON_VOTER last_known_addr { host: "127.0.253.252" port: 41893 } attrs { promote: true } } }
I20260812 16:37:32.679291 13731 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [sys.catalog]: This master's current role is: LEARNER
I20260812 16:37:32.680116 13758 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e: acquiring CA information for follower catalog manager: success
I20260812 16:37:32.680469 13758 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 16:37:32.707794 13575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:32.708015 13575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:32.708041 13575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:32.709182 13575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:32.709219 13575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:32.709245 13575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.193
I20260812 16:37:32.710707 13575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:40743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.193.keytab
--principal=oryx/127.0.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.193:36583
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.253.193
--webserver_port=39529
--tserver_master_addrs=127.0.253.254:39023,127.0.253.253:43719,127.0.253.252:41893
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.193
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:32.710913 13575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:32.711159 13575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:32.716346 13774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:32.716331 13775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:32.716322 13778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:32.716521 13575 server_base.cc:1034] running on GCE node
I20260812 16:37:32.716836 13575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:32.717036 13575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:32.718211 13575 hybrid_clock.cc:648] HybridClock initialized: now 1786552652718191 us; error 35 us; skew 500 ppm
Aug 12 16:37:32 dist-test-slave-w4v5 krb5kdc[12180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552652, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:32.721035 13575 init.cc:377] Logged in from keytab as oryx/127.0.253.193@KRBTEST.COM (short username oryx)
I20260812 16:37:32.721515 13575 webserver.cc:460] Webserver started at http://127.0.253.193:39529/ using document root <none> and password file <none>
I20260812 16:37:32.721762 13575 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:32.721856 13575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:32.723297 13575 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:32.723960 13789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:32.724140 13575 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:32.724251 13575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-0/wal
uuid: "826e312a29ed44e286fbd752c626acfd"
format_stamp: "Formatted at 2026-08-12 16:37:31 on dist-test-slave-w4v5"
server_key: "922b33a39de1077ba034028bfbaca9c9"
server_key_iv: "d3e1db71ea1bff1353d0af504e6e16d8"
server_key_version: "encryptionkey@0"
I20260812 16:37:32.724594 13575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:32.738737 13575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:32.741160 13575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:32.741329 13575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:32.741572 13575 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:37:32.741905 13575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:32.741966 13575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:32.742018 13575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:32.742048 13575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:32.748522 13575 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.193:36583
I20260812 16:37:32.748603 13967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.193:36583 every 8 connection(s)
I20260812 16:37:32.748955 13575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
I20260812 16:37:32.750839 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 13575
I20260812 16:37:32.750993 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 12579
Aug 12 16:37:32 dist-test-slave-w4v5 krb5kdc[12180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552652, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.253.193@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM
Aug 12 16:37:32 dist-test-slave-w4v5 krb5kdc[12180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552652, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.253.193@KRBTEST.COM for oryx/127.0.253.252@KRBTEST.COM
Aug 12 16:37:32 dist-test-slave-w4v5 krb5kdc[12180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552652, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.253.193@KRBTEST.COM for oryx/127.0.253.253@KRBTEST.COM
I20260812 16:37:32.771852 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.194:41387
--local_ip_for_outbound_sockets=127.0.253.194
--tserver_master_addrs=127.0.253.254:39023
--webserver_port=41153
--webserver_interface=127.0.253.194
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.194.keytab
--principal=oryx/127.0.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:40743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.0.253.254:39023,127.0.253.253:43719
--tserver_master_addrs=127.0.253.254:39023,127.0.253.253:43719,127.0.253.252:41893 with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:37:32.772480 13397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.749857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:51269 (local address 127.0.253.253:43719)
0812 16:37:32.749903 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:32.749905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:32.750221 (+ 316us) server_negotiation.cc:407] Connection header received
0812 16:37:32.761603 (+ 11382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:32.761605 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:32.761620 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:32.761654 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:32.762851 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.763652 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.766249 (+ 2597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.766575 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.766618 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.768850 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:32.768852 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:32.768852 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:32.768860 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:32.770633 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.771287 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.771288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.771288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.771351 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.771667 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.771668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.771668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.771841 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:32.771896 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:32.772293 (+ 397us) server_negotiation.cc:299] Negotiation successful
0812 16:37:32.772388 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 16:37:32.772526 12320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.749850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:40703 (local address 127.0.253.254:39023)
0812 16:37:32.749898 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:32.749900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:32.750221 (+ 321us) server_negotiation.cc:407] Connection header received
0812 16:37:32.761294 (+ 11073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:32.761296 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:32.761314 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:32.761347 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:32.762491 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.763193 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.764715 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.765025 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.765069 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.768776 (+ 3707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:32.768777 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:32.768777 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:32.768786 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:32.770542 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.771274 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.771275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.771276 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.771339 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.771676 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.771677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.771678 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.771835 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:32.771892 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:32.772290 (+ 398us) server_negotiation.cc:299] Negotiation successful
0812 16:37:32.772392 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 16:37:32.773123 13975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.749832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.253.193:40703)
0812 16:37:32.750070 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:32.750123 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:32.761129 (+ 11006us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:32.761546 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:32.761552 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:32.762099 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:32.762377 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.762382 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.763332 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.763333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.764608 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.764611 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.765181 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.765182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.765323 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.765913 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:32.765921 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:32.768654 (+ 2733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:32.770645 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.770655 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.770658 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.771184 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.771412 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.771413 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.771413 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.771582 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.771932 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:32.771939 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:32.772124 (+ 185us) client_negotiation.cc:769] Sending connection context
0812 16:37:32.772699 (+ 575us) client_negotiation.cc:240] Negotiation successful
0812 16:37:32.772896 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":164,"thread_start_us":82,"threads_started":1}
I20260812 16:37:32.773131 13973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.749832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.253:43719 (local address 127.0.253.193:51269)
0812 16:37:32.750070 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:32.750123 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:32.761116 (+ 10993us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:32.761730 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:32.761731 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:32.762076 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:32.762378 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.762385 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.763761 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.763762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.766150 (+ 2388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.766153 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.766700 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.766701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.766777 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.767252 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:32.767257 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:32.768760 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:32.770700 (+ 1940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.770701 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.770702 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.771186 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.771428 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.771429 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.771430 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.771582 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.771931 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:32.771938 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:32.772128 (+ 190us) client_negotiation.cc:769] Sending connection context
0812 16:37:32.772600 (+ 472us) client_negotiation.cc:240] Negotiation successful
0812 16:37:32.772823 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":2,"thread_start_us":113,"threads_started":1}
I20260812 16:37:32.773772 13968 heartbeater.cc:347] Connected to a master server at 127.0.253.254:39023
I20260812 16:37:32.773841 13970 heartbeater.cc:347] Connected to a master server at 127.0.253.253:43719
I20260812 16:37:32.773890 13970 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:32.773890 13968 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:32.774120 13978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.750302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:36545 (local address 127.0.253.252:41893)
0812 16:37:32.750473 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:32.750475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:32.750497 (+ 22us) server_negotiation.cc:407] Connection header received
0812 16:37:32.761428 (+ 10931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:32.761430 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:32.761450 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:32.761511 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:32.762831 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.763845 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.765207 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.765512 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.765580 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.768476 (+ 2896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:32.768478 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:32.768478 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:32.768486 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:32.771457 (+ 2971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.771847 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.771848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.771848 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.771908 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.772380 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.772381 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.772381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.772516 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:32.772969 (+ 453us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:32.773740 (+ 771us) server_negotiation.cc:299] Negotiation successful
0812 16:37:32.773995 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":61,"threads_started":1}
I20260812 16:37:32.774120 13970 heartbeater.cc:511] Master 127.0.253.253:43719 requested a full tablet report, sending...
I20260812 16:37:32.774120 13968 heartbeater.cc:511] Master 127.0.253.254:39023 requested a full tablet report, sending...
I20260812 16:37:32.774561 12253 ts_manager.cc:194] Re-registered known tserver with Master: 826e312a29ed44e286fbd752c626acfd (127.0.253.193:36583)
I20260812 16:37:32.774803 13976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.750231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:41893 (local address 127.0.253.193:36545)
0812 16:37:32.750353 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:32.750363 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:32.761251 (+ 10888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:32.762491 (+ 1240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:32.762492 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:32.762652 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:32.762733 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.762735 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.764008 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.764009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.765118 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.765120 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.765637 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.765638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.765685 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.766123 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:32.766128 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:32.768366 (+ 2238us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:32.771538 (+ 3172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.771539 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.771539 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.771769 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.772246 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.772247 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.772248 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.772311 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.772999 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:32.773000 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:32.773043 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 16:37:32.773107 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 16:37:32.774420 (+ 1313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":68,"threads_started":1}
I20260812 16:37:32.775167 12253 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.253.193@KRBTEST.COM'} at 127.0.253.193:40703
I20260812 16:37:32.775388 13308 ts_manager.cc:194] Registered new tserver with Master: 826e312a29ed44e286fbd752c626acfd (127.0.253.193:36583)
I20260812 16:37:32.775483 13969 heartbeater.cc:347] Connected to a master server at 127.0.253.252:41893
I20260812 16:37:32.775543 13969 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:32.775616 13969 heartbeater.cc:511] Master 127.0.253.252:41893 requested a full tablet report, sending...
I20260812 16:37:32.776360 13652 ts_manager.cc:194] Registered new tserver with Master: 826e312a29ed44e286fbd752c626acfd (127.0.253.193:36583)
W20260812 16:37:32.860580 13980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:32.860811 13980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:32.860852 13980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:32.862026 13980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:32.862090 13980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:32.862134 13980 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.194
I20260812 16:37:32.863736 13980 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:40743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.194.keytab
--principal=oryx/127.0.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.194:41387
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.253.194
--webserver_port=41153
--tserver_master_addrs=127.0.253.254:39023,127.0.253.253:43719,127.0.253.252:41893
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13980
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.194
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:32.863991 13980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:32.864228 13980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:32.869473 13986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:32.869493 13985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:32.869537 13980 server_base.cc:1034] running on GCE node
W20260812 16:37:32.869710 13988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:32.869946 13980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:32.870201 13980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:32.871383 13980 hybrid_clock.cc:648] HybridClock initialized: now 1786552652871371 us; error 29 us; skew 500 ppm
Aug 12 16:37:32 dist-test-slave-w4v5 krb5kdc[12180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552652, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:32.874765 13980 init.cc:377] Logged in from keytab as oryx/127.0.253.194@KRBTEST.COM (short username oryx)
I20260812 16:37:32.875252 13980 webserver.cc:460] Webserver started at http://127.0.253.194:41153/ using document root <none> and password file <none>
I20260812 16:37:32.875442 13980 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:32.875497 13980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:32.876955 13980 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:32.877599 13998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:32.877851 13980 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:32.877949 13980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-1/wal
uuid: "8b99632ff7e84f99a05e3e8bd6435268"
format_stamp: "Formatted at 2026-08-12 16:37:31 on dist-test-slave-w4v5"
server_key: "2416942370e53804ce8c95b58f9c9847"
server_key_iv: "b14fc5fd89a1d1cb05eb6a44ffe0b0b8"
server_key_version: "encryptionkey@0"
I20260812 16:37:32.878324 13980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:32.895887 13980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:32.898381 13980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:32.898546 13980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:32.898852 13980 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:37:32.899276 13980 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:32.899314 13980 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:32.899356 13980 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:32.899385 13980 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:32.905580 13980 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.194:41387
I20260812 16:37:32.905719 14188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.194:41387 every 8 connection(s)
I20260812 16:37:32.906113 13980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
I20260812 16:37:32.909854 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 13980
I20260812 16:37:32.910017 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 12827
Aug 12 16:37:32 dist-test-slave-w4v5 krb5kdc[12180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552652, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.253.194@KRBTEST.COM for oryx/127.0.253.252@KRBTEST.COM
Aug 12 16:37:32 dist-test-slave-w4v5 krb5kdc[12180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552652, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.253.194@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM
I20260812 16:37:32.917474 13978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.907316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:50417 (local address 127.0.253.252:41893)
0812 16:37:32.907367 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:32.907369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:32.908349 (+ 980us) server_negotiation.cc:407] Connection header received
0812 16:37:32.908403 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:32.908404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:32.908416 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:32.908480 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:32.909018 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.909768 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.911029 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.911341 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.911379 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.914164 (+ 2785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:32.914166 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:32.914166 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:32.914174 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:32.915713 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.916136 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.916138 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.916138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.916204 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.916591 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.916592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.916593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.916726 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:32.916817 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:32.917252 (+ 435us) server_negotiation.cc:299] Negotiation successful
0812 16:37:32.917350 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 16:37:32.917837 14203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.906926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:41893 (local address 127.0.253.194:50417)
0812 16:37:32.908265 (+ 1339us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:32.908276 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:32.908340 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:32.908483 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:32.908484 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:32.908759 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:32.908874 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.908876 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.909852 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.909852 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.910939 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.910942 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.911457 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.911458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.911584 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.912095 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:32.912104 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:32.914055 (+ 1951us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:32.915810 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.915819 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.915821 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.916055 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.916401 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.916401 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.916402 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.916509 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.916812 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:32.916822 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:32.917048 (+ 226us) client_negotiation.cc:769] Sending connection context
0812 16:37:32.917204 (+ 156us) client_negotiation.cc:240] Negotiation successful
0812 16:37:32.917440 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1315,"mutex_wait_us":27,"thread_start_us":135,"threads_started":1}
I20260812 16:37:32.918443 14191 heartbeater.cc:347] Connected to a master server at 127.0.253.252:41893
I20260812 16:37:32.918561 14191 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:32.918819 14191 heartbeater.cc:511] Master 127.0.253.252:41893 requested a full tablet report, sending...
I20260812 16:37:32.919351 13652 ts_manager.cc:194] Registered new tserver with Master: 8b99632ff7e84f99a05e3e8bd6435268 (127.0.253.194:41387)
I20260812 16:37:32.919440 12320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.907243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:48353 (local address 127.0.253.254:39023)
0812 16:37:32.907392 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:32.907394 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:32.907405 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:37:32.907972 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:32.907973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:32.907985 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:32.908051 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:32.908909 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.909711 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.912356 (+ 2645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.912814 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.912866 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.915866 (+ 3000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:32.915868 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:32.915869 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:32.915878 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:32.917653 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.918446 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.918447 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.918448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.918526 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.918795 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.918796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.918797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.918944 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:32.919022 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:32.919252 (+ 230us) server_negotiation.cc:299] Negotiation successful
0812 16:37:32.919339 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260812 16:37:32.919507 14202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.906926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.253.194:48353)
0812 16:37:32.907224 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:32.907267 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:32.907861 (+ 594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:32.908146 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:32.908150 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:32.908592 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:32.908812 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.908816 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.909806 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.909807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.912232 (+ 2425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.912236 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.913202 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.913204 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.913287 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.913756 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:32.913763 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:32.915748 (+ 1985us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:32.918046 (+ 2298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.918047 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.918047 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.918260 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.918619 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.918621 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.918621 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.918712 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.919028 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:32.919030 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:32.919110 (+ 80us) client_negotiation.cc:769] Sending connection context
0812 16:37:32.919201 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 16:37:32.919334 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":1297,"thread_start_us":136,"threads_started":1}
I20260812 16:37:32.919981 14190 heartbeater.cc:347] Connected to a master server at 127.0.253.254:39023
I20260812 16:37:32.920053 14190 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:32.920128 14190 heartbeater.cc:511] Master 127.0.253.254:39023 requested a full tablet report, sending...
I20260812 16:37:32.920392 12253 ts_manager.cc:194] Re-registered known tserver with Master: 8b99632ff7e84f99a05e3e8bd6435268 (127.0.253.194:41387)
I20260812 16:37:32.920785 12253 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.253.194@KRBTEST.COM'} at 127.0.253.194:48353
Aug 12 16:37:32 dist-test-slave-w4v5 krb5kdc[12180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552652, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.253.194@KRBTEST.COM for oryx/127.0.253.253@KRBTEST.COM
I20260812 16:37:32.943041 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.195:43725
--local_ip_for_outbound_sockets=127.0.253.195
--tserver_master_addrs=127.0.253.254:39023
--webserver_port=36813
--webserver_interface=127.0.253.195
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.195.keytab
--principal=oryx/127.0.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:40743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.0.253.254:39023,127.0.253.253:43719
--tserver_master_addrs=127.0.253.254:39023,127.0.253.253:43719,127.0.253.252:41893 with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:37:32.944875 14198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.906925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.253:43719 (local address 127.0.253.194:56837)
0812 16:37:32.907470 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:32.907479 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:32.907871 (+ 392us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:32.908146 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:32.908151 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:32.908759 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:32.908874 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.908876 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.909647 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.909648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.911122 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:32.911126 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.939175 (+ 28049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.939176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:32.939264 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.939810 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:32.939816 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:32.941777 (+ 1961us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:32.943661 (+ 1884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.943662 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.943663 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.943916 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.944123 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:32.944124 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:32.944125 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:32.944205 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:32.944442 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:32.944443 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:32.944496 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 16:37:32.944568 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 16:37:32.944723 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"mutex_wait_us":51,"thread_start_us":151,"threads_started":1}
I20260812 16:37:32.944880 13397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:32.907279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:56837 (local address 127.0.253.253:43719)
0812 16:37:32.907324 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:32.907326 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:32.907547 (+ 221us) server_negotiation.cc:407] Connection header received
0812 16:37:32.907971 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:32.907972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:32.907985 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:32.908045 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:32.909044 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.909534 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.938583 (+ 29049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:32.938963 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:32.939104 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:32.941896 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:32.941898 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:32.941899 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:32.941907 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:32.943535 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.943996 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.943997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.943997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.944051 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:32.944266 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:32.944267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:32.944267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:32.944378 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:32.944428 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:32.944596 (+ 168us) server_negotiation.cc:299] Negotiation successful
0812 16:37:32.944752 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 16:37:32.945462 14193 heartbeater.cc:347] Connected to a master server at 127.0.253.253:43719
I20260812 16:37:32.945545 14193 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:32.945662 14193 heartbeater.cc:511] Master 127.0.253.253:43719 requested a full tablet report, sending...
I20260812 16:37:32.946061 13308 ts_manager.cc:194] Registered new tserver with Master: 8b99632ff7e84f99a05e3e8bd6435268 (127.0.253.194:41387)
W20260812 16:37:33.029492 14204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:33.029702 14204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:33.029727 14204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:33.030853 14204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:33.030890 14204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:33.030916 14204 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.195
I20260812 16:37:33.032433 14204 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:40743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.195.keytab
--principal=oryx/127.0.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.195:43725
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.253.195
--webserver_port=36813
--tserver_master_addrs=127.0.253.254:39023,127.0.253.253:43719,127.0.253.252:41893
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.195
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:33.032630 14204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:33.032811 14204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:33.038105 14215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:33.038089 14213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:33.038241 14204 server_base.cc:1034] running on GCE node
W20260812 16:37:33.038374 14212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:33.038617 14204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:33.038913 14204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:33.040093 14204 hybrid_clock.cc:648] HybridClock initialized: now 1786552653040079 us; error 42 us; skew 500 ppm
Aug 12 16:37:33 dist-test-slave-w4v5 krb5kdc[12180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552653, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:33.042972 14204 init.cc:377] Logged in from keytab as oryx/127.0.253.195@KRBTEST.COM (short username oryx)
I20260812 16:37:33.043460 14204 webserver.cc:460] Webserver started at http://127.0.253.195:36813/ using document root <none> and password file <none>
I20260812 16:37:33.043764 14204 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:33.043861 14204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:33.045233 14204 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 16:37:33.045862 14224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:33.046064 14204 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:37:33.046160 14204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-2/wal
uuid: "0af3a5dcce9d4ae890aecc4f9f4ff6f7"
format_stamp: "Formatted at 2026-08-12 16:37:32 on dist-test-slave-w4v5"
server_key: "f1c1c5e7eb44cdcee06d90b8585f7b3b"
server_key_iv: "8ab88f0ab39abb4a437c5f85846f417f"
server_key_version: "encryptionkey@0"
I20260812 16:37:33.046528 14204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:33.058938 14204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:33.061124 14204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:33.061295 14204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:33.061534 14204 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:37:33.061856 14204 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:33.061914 14204 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:33.061959 14204 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:33.061986 14204 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:33.068011 14204 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.195:43725
I20260812 16:37:33.068073 14420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.195:43725 every 8 connection(s)
I20260812 16:37:33.068410 14204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
I20260812 16:37:33.069301 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 14204
Aug 12 16:37:33 dist-test-slave-w4v5 krb5kdc[12180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552653, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.253.195@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM
Aug 12 16:37:33 dist-test-slave-w4v5 krb5kdc[12180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552653, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.253.195@KRBTEST.COM for oryx/127.0.253.252@KRBTEST.COM
Aug 12 16:37:33 dist-test-slave-w4v5 krb5kdc[12180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552653, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.253.195@KRBTEST.COM for oryx/127.0.253.253@KRBTEST.COM
I20260812 16:37:33.083568 12320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:33.069814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:45093 (local address 127.0.253.254:39023)
0812 16:37:33.069866 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:33.069868 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:33.069878 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:37:33.070263 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:33.070265 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:33.070280 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:33.070358 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:33.071509 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.072281 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.073617 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.073912 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.073968 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:33.079196 (+ 5228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:33.079198 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:33.079198 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:33.079206 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:33.081765 (+ 2559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:33.082329 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:33.082330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:33.082330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:33.082393 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:33.082694 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:33.082695 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:33.082696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:33.082856 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:33.082918 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:33.083375 (+ 457us) server_negotiation.cc:299] Negotiation successful
0812 16:37:33.083458 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 16:37:33.083561 14449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:33.070876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38772 (local address 127.0.253.253:43719)
0812 16:37:33.072144 (+ 1268us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:33.072146 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:33.072152 (+ 6us) server_negotiation.cc:407] Connection header received
0812 16:37:33.072172 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:33.072172 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:33.072186 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:33.072241 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:33.075834 (+ 3593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.076553 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.077567 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.077914 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.077960 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:33.080322 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:33.080323 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:33.080324 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:33.080330 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:33.082261 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:33.082655 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:33.082656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:33.082657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:33.082716 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:33.082924 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:33.082925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:33.082926 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:33.083057 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:33.083169 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:33.083375 (+ 206us) server_negotiation.cc:299] Negotiation successful
0812 16:37:33.083459 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1249,"thread_start_us":50,"threads_started":1}
I20260812 16:37:33.083753 14432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:33.069171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.253:43719 (local address 127.0.253.195:53877)
0812 16:37:33.069607 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:33.069620 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:33.070145 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:33.070599 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:33.070600 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:33.071142 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:33.071390 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:33.071398 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.072138 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.072139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:33.073534 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:33.073608 (+ 74us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.074097 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.074098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:33.074195 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:33.075570 (+ 1375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:33.075576 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:33.078743 (+ 3167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:33.081782 (+ 3039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:33.081791 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:33.081793 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:33.082249 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:33.082495 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:33.082496 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:33.082496 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:33.082622 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:33.082990 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:33.082991 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:33.083167 (+ 176us) client_negotiation.cc:769] Sending connection context
0812 16:37:33.083283 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 16:37:33.083498 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"mutex_wait_us":56,"thread_start_us":100,"threads_started":1}
I20260812 16:37:33.084208 13397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:33.069172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:53877 (local address 127.0.253.253:43719)
0812 16:37:33.069233 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:33.069235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:33.069683 (+ 448us) server_negotiation.cc:407] Connection header received
0812 16:37:33.070427 (+ 744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:33.070429 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:33.070440 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:33.070523 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:33.071525 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.072024 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.073690 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.073989 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.074047 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:33.078841 (+ 4794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:33.078842 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:33.078843 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:33.078851 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:33.080795 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:33.082345 (+ 1550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:33.082347 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:33.082347 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:33.082413 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:33.082703 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:33.082704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:33.082705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:33.082870 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:33.082932 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:33.084041 (+ 1109us) server_negotiation.cc:299] Negotiation successful
0812 16:37:33.084121 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 16:37:33.084335 14450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:33.071121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54328 (local address 127.0.253.254:39023)
0812 16:37:33.071417 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:33.071419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:33.073526 (+ 2107us) server_negotiation.cc:407] Connection header received
0812 16:37:33.073551 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:33.073552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:33.073564 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:33.073632 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:33.074314 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.076108 (+ 1794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.077390 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.077712 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.077787 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:33.080976 (+ 3189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:33.080977 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:33.080978 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:33.080986 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:33.082850 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:33.083267 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:33.083268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:33.083269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:33.083331 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:33.083537 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:33.083538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:33.083539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:33.083667 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:33.083751 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:33.084156 (+ 405us) server_negotiation.cc:299] Negotiation successful
0812 16:37:33.084232 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":101,"threads_started":1}
I20260812 16:37:33.084685 14434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:33.069702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.253.195:45093)
0812 16:37:33.069827 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:33.069836 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:33.070145 (+ 309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:33.070847 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:33.070848 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:33.071163 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:33.071390 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:33.071396 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.072369 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.072370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:33.073512 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:33.073515 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.074034 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.074035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:33.074195 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:33.074848 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:33.074858 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:33.079121 (+ 4263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:33.081855 (+ 2734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:33.081856 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:33.081857 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:33.082239 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:33.082473 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:33.082474 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:33.082474 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:33.082621 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:33.082965 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:33.082972 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:33.083167 (+ 195us) client_negotiation.cc:769] Sending connection context
0812 16:37:33.083282 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 16:37:33.084614 (+ 1332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":1214,"thread_start_us":63,"threads_started":1}
I20260812 16:37:33.084666 14426 heartbeater.cc:347] Connected to a master server at 127.0.253.253:43719
I20260812 16:37:33.084813 14426 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:33.085033 14426 heartbeater.cc:511] Master 127.0.253.253:43719 requested a full tablet report, sending...
I20260812 16:37:33.085577 14421 heartbeater.cc:347] Connected to a master server at 127.0.253.254:39023
I20260812 16:37:33.085630 14421 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:33.085695 14421 heartbeater.cc:511] Master 127.0.253.254:39023 requested a full tablet report, sending...
I20260812 16:37:33.085915 13308 ts_manager.cc:194] Registered new tserver with Master: 0af3a5dcce9d4ae890aecc4f9f4ff6f7 (127.0.253.195:43725)
I20260812 16:37:33.086211 12253 ts_manager.cc:194] Re-registered known tserver with Master: 0af3a5dcce9d4ae890aecc4f9f4ff6f7 (127.0.253.195:43725)
I20260812 16:37:33.086642 12253 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.253.195@KRBTEST.COM'} at 127.0.253.195:45093
I20260812 16:37:33.087380 14448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:33.070638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35120 (local address 127.0.253.252:41893)
0812 16:37:33.072574 (+ 1936us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:33.072576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:33.072583 (+ 7us) server_negotiation.cc:407] Connection header received
0812 16:37:33.072602 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:33.072603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:33.072613 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:33.072668 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:33.073426 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.074216 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.075289 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.076281 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.076350 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:33.078367 (+ 2017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:33.078368 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:33.078369 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:33.078375 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:33.085923 (+ 7548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:33.086390 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:33.086391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:33.086391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:33.086479 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:33.086749 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:33.086750 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:33.086750 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:33.086853 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:33.086955 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:33.087149 (+ 194us) server_negotiation.cc:299] Negotiation successful
0812 16:37:33.087278 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":1917,"thread_start_us":90,"threads_started":1}
I20260812 16:37:33.087484 13978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:33.069452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:40095 (local address 127.0.253.252:41893)
0812 16:37:33.069499 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:33.069501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:33.069511 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:37:33.070384 (+ 873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:33.070385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:33.070398 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:33.070429 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:33.071508 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.072273 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.074494 (+ 2221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.074824 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.074893 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:33.079529 (+ 4636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:33.079531 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:33.079547 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:33.079556 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:33.081637 (+ 2081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:33.086398 (+ 4761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:33.086399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:33.086399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:33.086500 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:33.086715 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:33.086716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:33.086716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:33.086852 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:33.086910 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:33.087332 (+ 422us) server_negotiation.cc:299] Negotiation successful
0812 16:37:33.087404 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":24}
I20260812 16:37:33.088071 14429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:33.069171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:41893 (local address 127.0.253.195:40095)
0812 16:37:33.069338 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:33.069380 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:33.070286 (+ 906us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:33.070534 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:33.070545 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:33.071047 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:33.071390 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:33.071396 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.072369 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.072370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:33.074415 (+ 2045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:33.074417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.074948 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.074949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:33.075022 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:33.075570 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:33.075576 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:33.079436 (+ 3860us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:33.086022 (+ 6586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:33.086023 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:33.086024 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:33.086268 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:33.086581 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:33.086582 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:33.086582 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:33.086640 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:33.087126 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:33.087127 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:33.087169 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 16:37:33.087655 (+ 486us) client_negotiation.cc:240] Negotiation successful
0812 16:37:33.087861 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":45,"thread_start_us":95,"threads_started":1}
I20260812 16:37:33.088423 14423 heartbeater.cc:347] Connected to a master server at 127.0.253.252:41893
I20260812 16:37:33.088480 14423 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:33.088542 14423 heartbeater.cc:511] Master 127.0.253.252:41893 requested a full tablet report, sending...
I20260812 16:37:33.088806 13652 ts_manager.cc:194] Registered new tserver with Master: 0af3a5dcce9d4ae890aecc4f9f4ff6f7 (127.0.253.195:43725)
I20260812 16:37:33.086257 12252 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54328:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:37:33.089267 12252 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:37:33.096900 13234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:33.092832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:41387 (local address 127.0.0.1:46318)
0812 16:37:33.092879 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:33.092890 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:33.092929 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:33.093368 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:33.093369 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:33.093371 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:33.093419 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:33.093421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.094375 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.094376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:33.095644 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:33.095647 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.096642 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.096643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:33.096703 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:33.096707 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:37:33.096780 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 16:37:33.096827 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260812 16:37:33.096911 14462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:33.092945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46318 (local address 127.0.253.194:41387)
0812 16:37:33.093157 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:33.093163 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:33.093175 (+ 12us) server_negotiation.cc:407] Connection header received
0812 16:37:33.093222 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:33.093225 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:33.093243 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:33.093334 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:33.093485 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.094292 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.095730 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.096550 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.096626 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:33.096702 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:33.096792 (+ 90us) server_negotiation.cc:299] Negotiation successful
0812 16:37:33.096830 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":82,"threads_started":1}
I20260812 16:37:33.097759 14083 tablet_service.cc:1467] Processing CreateTablet for tablet 325b1ff6466f4d1bbb4a5889a51b096a (DEFAULT_TABLE table=test-table [id=55c9e9a8a3fd4838a57d58d8837ebd06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:33.098210 14083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 325b1ff6466f4d1bbb4a5889a51b096a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:33.098294 14461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:33.093024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:36583 (local address 127.0.0.1:36850)
0812 16:37:33.094398 (+ 1374us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:33.094407 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:33.094449 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:33.094669 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:33.094670 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:33.094672 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:33.094717 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:33.094718 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.095956 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.095957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:33.097050 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:33.097053 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.098050 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.098051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:33.098115 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:33.098120 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:37:33.098169 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 16:37:33.098218 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1360,"thread_start_us":97,"threads_started":1}
I20260812 16:37:33.098387 14463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:33.092939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44418 (local address 127.0.253.195:43725)
0812 16:37:33.094545 (+ 1606us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:33.094548 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:33.094556 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:37:33.094583 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:33.094584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:33.094596 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:33.094698 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:33.094846 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.095783 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.096808 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.098024 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.098106 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:33.098180 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:33.098266 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 16:37:33.098302 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1577,"thread_start_us":93,"threads_started":1}
I20260812 16:37:33.098559 14467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:33.093160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36850 (local address 127.0.253.193:36583)
0812 16:37:33.093348 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:33.093353 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:33.094481 (+ 1128us) server_negotiation.cc:407] Connection header received
0812 16:37:33.094537 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:33.094539 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:33.094558 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:33.094654 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:33.094785 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.095877 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.097144 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.097850 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.097923 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:33.097995 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:33.098401 (+ 406us) server_negotiation.cc:299] Negotiation successful
0812 16:37:33.098453 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":114,"threads_started":1}
I20260812 16:37:33.098824 14464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:33.092845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.195:43725 (local address 127.0.0.1:44418)
0812 16:37:33.093185 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:33.093196 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:33.093249 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:33.094756 (+ 1507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:33.094756 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:33.094758 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:33.094793 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:33.094794 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.095864 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.095865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:33.096718 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:33.096719 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.098107 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.098108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:33.098177 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:33.098182 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:37:33.098251 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 16:37:33.098759 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":40,"thread_start_us":91,"threads_started":1}
I20260812 16:37:33.099478 14299 tablet_service.cc:1467] Processing CreateTablet for tablet 325b1ff6466f4d1bbb4a5889a51b096a (DEFAULT_TABLE table=test-table [id=55c9e9a8a3fd4838a57d58d8837ebd06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:33.099448 13864 tablet_service.cc:1467] Processing CreateTablet for tablet 325b1ff6466f4d1bbb4a5889a51b096a (DEFAULT_TABLE table=test-table [id=55c9e9a8a3fd4838a57d58d8837ebd06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:33.099833 13864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 325b1ff6466f4d1bbb4a5889a51b096a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:33.099833 14299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 325b1ff6466f4d1bbb4a5889a51b096a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:33.101491 14469 tablet_bootstrap.cc:492] T 325b1ff6466f4d1bbb4a5889a51b096a P 8b99632ff7e84f99a05e3e8bd6435268: Bootstrap starting.
I20260812 16:37:33.102094 14469 tablet_bootstrap.cc:654] T 325b1ff6466f4d1bbb4a5889a51b096a P 8b99632ff7e84f99a05e3e8bd6435268: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:33.102329 14469 log.cc:824] T 325b1ff6466f4d1bbb4a5889a51b096a P 8b99632ff7e84f99a05e3e8bd6435268: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:33.102990 14469 tablet_bootstrap.cc:492] T 325b1ff6466f4d1bbb4a5889a51b096a P 8b99632ff7e84f99a05e3e8bd6435268: No bootstrap required, opened a new log
I20260812 16:37:33.103037 14469 ts_tablet_manager.cc:1397] T 325b1ff6466f4d1bbb4a5889a51b096a P 8b99632ff7e84f99a05e3e8bd6435268: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:37:33.103446 14472 tablet_bootstrap.cc:492] T 325b1ff6466f4d1bbb4a5889a51b096a P 826e312a29ed44e286fbd752c626acfd: Bootstrap starting.
I20260812 16:37:33.104118 14472 tablet_bootstrap.cc:654] T 325b1ff6466f4d1bbb4a5889a51b096a P 826e312a29ed44e286fbd752c626acfd: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:33.104445 14472 log.cc:824] T 325b1ff6466f4d1bbb4a5889a51b096a P 826e312a29ed44e286fbd752c626acfd: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:33.104799 14475 tablet_bootstrap.cc:492] T 325b1ff6466f4d1bbb4a5889a51b096a P 0af3a5dcce9d4ae890aecc4f9f4ff6f7: Bootstrap starting.
I20260812 16:37:33.105249 14472 tablet_bootstrap.cc:492] T 325b1ff6466f4d1bbb4a5889a51b096a P 826e312a29ed44e286fbd752c626acfd: No bootstrap required, opened a new log
I20260812 16:37:33.105338 14472 ts_tablet_manager.cc:1397] T 325b1ff6466f4d1bbb4a5889a51b096a P 826e312a29ed44e286fbd752c626acfd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:37:33.105536 14475 tablet_bootstrap.cc:654] T 325b1ff6466f4d1bbb4a5889a51b096a P 0af3a5dcce9d4ae890aecc4f9f4ff6f7: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:33.105904 14475 log.cc:824] T 325b1ff6466f4d1bbb4a5889a51b096a P 0af3a5dcce9d4ae890aecc4f9f4ff6f7: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:33.106308 14469 raft_consensus.cc:348] T 325b1ff6466f4d1bbb4a5889a51b096a P 8b99632ff7e84f99a05e3e8bd6435268 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b99632ff7e84f99a05e3e8bd6435268" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 41387 } } peers { permanent_uuid: "0af3a5dcce9d4ae890aecc4f9f4ff6f7" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43725 } } peers { permanent_uuid: "826e312a29ed44e286fbd752c626acfd" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 36583 } }
I20260812 16:37:33.106439 14469 raft_consensus.cc:374] T 325b1ff6466f4d1bbb4a5889a51b096a P 8b99632ff7e84f99a05e3e8bd6435268 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:33.106462 14469 raft_consensus.cc:729] T 325b1ff6466f4d1bbb4a5889a51b096a P 8b99632ff7e84f99a05e3e8bd6435268 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b99632ff7e84f99a05e3e8bd6435268, State: Initialized, Role: FOLLOWER
I20260812 16:37:33.106585 14469 consensus_queue.cc:260] T 325b1ff6466f4d1bbb4a5889a51b096a P 8b99632ff7e84f99a05e3e8bd6435268 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b99632ff7e84f99a05e3e8bd6435268" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 41387 } } peers { permanent_uuid: "0af3a5dcce9d4ae890aecc4f9f4ff6f7" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43725 } } peers { permanent_uuid: "826e312a29ed44e286fbd752c626acfd" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 36583 } }
I20260812 16:37:33.106863 14190 heartbeater.cc:503] Master 127.0.253.254:39023 was elected leader, sending a full tablet report...
I20260812 16:37:33.106894 14469 ts_tablet_manager.cc:1428] T 325b1ff6466f4d1bbb4a5889a51b096a P 8b99632ff7e84f99a05e3e8bd6435268: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260812 16:37:33.107429 14472 raft_consensus.cc:348] T 325b1ff6466f4d1bbb4a5889a51b096a P 826e312a29ed44e286fbd752c626acfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b99632ff7e84f99a05e3e8bd6435268" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 41387 } } peers { permanent_uuid: "0af3a5dcce9d4ae890aecc4f9f4ff6f7" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43725 } } peers { permanent_uuid: "826e312a29ed44e286fbd752c626acfd" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 36583 } }
I20260812 16:37:33.107558 14475 tablet_bootstrap.cc:492] T 325b1ff6466f4d1bbb4a5889a51b096a P 0af3a5dcce9d4ae890aecc4f9f4ff6f7: No bootstrap required, opened a new log
I20260812 16:37:33.107591 14472 raft_consensus.cc:374] T 325b1ff6466f4d1bbb4a5889a51b096a P 826e312a29ed44e286fbd752c626acfd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:33.107621 14475 ts_tablet_manager.cc:1397] T 325b1ff6466f4d1bbb4a5889a51b096a P 0af3a5dcce9d4ae890aecc4f9f4ff6f7: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 16:37:33.107630 14472 raft_consensus.cc:729] T 325b1ff6466f4d1bbb4a5889a51b096a P 826e312a29ed44e286fbd752c626acfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 826e312a29ed44e286fbd752c626acfd, State: Initialized, Role: FOLLOWER
I20260812 16:37:33.107749 14472 consensus_queue.cc:260] T 325b1ff6466f4d1bbb4a5889a51b096a P 826e312a29ed44e286fbd752c626acfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b99632ff7e84f99a05e3e8bd6435268" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 41387 } } peers { permanent_uuid: "0af3a5dcce9d4ae890aecc4f9f4ff6f7" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43725 } } peers { permanent_uuid: "826e312a29ed44e286fbd752c626acfd" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 36583 } }
I20260812 16:37:33.107993 14472 ts_tablet_manager.cc:1428] T 325b1ff6466f4d1bbb4a5889a51b096a P 826e312a29ed44e286fbd752c626acfd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:37:33.108137 13968 heartbeater.cc:503] Master 127.0.253.254:39023 was elected leader, sending a full tablet report...
I20260812 16:37:33.110566 14475 raft_consensus.cc:348] T 325b1ff6466f4d1bbb4a5889a51b096a P 0af3a5dcce9d4ae890aecc4f9f4ff6f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b99632ff7e84f99a05e3e8bd6435268" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 41387 } } peers { permanent_uuid: "0af3a5dcce9d4ae890aecc4f9f4ff6f7" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43725 } } peers { permanent_uuid: "826e312a29ed44e286fbd752c626acfd" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 36583 } }
I20260812 16:37:33.110734 14475 raft_consensus.cc:374] T 325b1ff6466f4d1bbb4a5889a51b096a P 0af3a5dcce9d4ae890aecc4f9f4ff6f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:33.110772 14475 raft_consensus.cc:729] T 325b1ff6466f4d1bbb4a5889a51b096a P 0af3a5dcce9d4ae890aecc4f9f4ff6f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0af3a5dcce9d4ae890aecc4f9f4ff6f7, State: Initialized, Role: FOLLOWER
I20260812 16:37:33.110872 14475 consensus_queue.cc:260] T 325b1ff6466f4d1bbb4a5889a51b096a P 0af3a5dcce9d4ae890aecc4f9f4ff6f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b99632ff7e84f99a05e3e8bd6435268" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 41387 } } peers { permanent_uuid: "0af3a5dcce9d4ae890aecc4f9f4ff6f7" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43725 } } peers { permanent_uuid: "826e312a29ed44e286fbd752c626acfd" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 36583 } }
I20260812 16:37:33.111158 14421 heartbeater.cc:503] Master 127.0.253.254:39023 was elected leader, sending a full tablet report...
I20260812 16:37:33.111174 14475 ts_tablet_manager.cc:1428] T 325b1ff6466f4d1bbb4a5889a51b096a P 0af3a5dcce9d4ae890aecc4f9f4ff6f7: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260812 16:37:33.150650 13978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:33.147892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35134 (local address 127.0.253.252:41893)
0812 16:37:33.147962 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:33.147964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:33.147973 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:37:33.148008 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:33.148009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:33.148029 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:33.148103 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:33.148249 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.148752 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.149828 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.150246 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.150323 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:33.150415 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:33.150534 (+ 119us) server_negotiation.cc:299] Negotiation successful
0812 16:37:33.150592 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 16:37:33.150635 14464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:33.147793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:41893 (local address 127.0.0.1:35134)
0812 16:37:33.147856 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:33.147874 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:33.147930 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:33.148108 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:33.148109 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:33.148111 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:33.148172 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:33.148174 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.148846 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.148848 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:33.149728 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:33.149731 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.150343 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.150344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:33.150407 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:33.150411 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:37:33.150480 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 16:37:33.150553 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 16:37:33.151119 13669 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260812 16:37:33.152081 13731 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 283116b420234f088056a1f917a31bb7. Latest consensus state: current_term: 1 leader_uuid: "283116b420234f088056a1f917a31bb7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: false } } peers { permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: NON_VOTER last_known_addr { host: "127.0.253.252" port: 41893 } attrs { promote: true } } }
I20260812 16:37:33.152204 13731 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [sys.catalog]: This master's current role is: LEARNER
W20260812 16:37:33.157356 14197 tablet.cc:2369] T 325b1ff6466f4d1bbb4a5889a51b096a P 8b99632ff7e84f99a05e3e8bd6435268: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:37:33.210129 14483 raft_consensus.cc:482] T 325b1ff6466f4d1bbb4a5889a51b096a P 826e312a29ed44e286fbd752c626acfd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:37:33.210253 14483 raft_consensus.cc:504] T 325b1ff6466f4d1bbb4a5889a51b096a P 826e312a29ed44e286fbd752c626acfd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b99632ff7e84f99a05e3e8bd6435268" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 41387 } } peers { permanent_uuid: "0af3a5dcce9d4ae890aecc4f9f4ff6f7" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43725 } } peers { permanent_uuid: "826e312a29ed44e286fbd752c626acfd" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 36583 } }
I20260812 16:37:33.210753 14483 leader_election.cc:288] T 325b1ff6466f4d1bbb4a5889a51b096a P 826e312a29ed44e286fbd752c626acfd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b99632ff7e84f99a05e3e8bd6435268 (127.0.253.194:41387), 0af3a5dcce9d4ae890aecc4f9f4ff6f7 (127.0.253.195:43725)
I20260812 16:37:33.214753 13975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:33.210907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.195:43725 (local address 127.0.253.193:42071)
0812 16:37:33.210958 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:33.210971 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:33.211018 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:33.211237 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:33.211238 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:33.211240 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:33.211326 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:33.211328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.212248 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.212249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:33.213621 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:33.213624 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.214446 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.214447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:33.214541 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:33.214545 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:37:33.214605 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 16:37:33.214655 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":86}
I20260812 16:37:33.214781 13976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:33.210908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:41387 (local address 127.0.253.193:48725)
0812 16:37:33.211041 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:33.211054 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:33.211125 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:33.211276 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:33.211277 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:33.211279 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:33.211350 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:33.211351 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.212250 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.212251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:33.213621 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:33.213624 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.214498 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.214498 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:33.214570 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:33.214573 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:37:33.214626 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 16:37:33.214656 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":155}
I20260812 16:37:33.214767 14463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:33.211067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:42071 (local address 127.0.253.195:43725)
0812 16:37:33.211140 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:33.211142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:33.211149 (+ 7us) server_negotiation.cc:407] Connection header received
0812 16:37:33.211170 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:33.211170 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:33.211178 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:33.211251 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:33.211404 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.212156 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.213748 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.214359 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.214448 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:33.214457 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:33.214647 (+ 190us) server_negotiation.cc:299] Negotiation successful
0812 16:37:33.214694 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 16:37:33.214774 14462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:33.211028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:48725 (local address 127.0.253.194:41387)
0812 16:37:33.211099 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:33.211101 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:33.211110 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:37:33.211184 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:33.211184 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:33.211194 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:33.211249 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:33.211456 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.212156 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.213781 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.214359 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.214420 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:33.214427 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:33.214646 (+ 219us) server_negotiation.cc:299] Negotiation successful
0812 16:37:33.214687 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 16:37:33.215140 14116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "325b1ff6466f4d1bbb4a5889a51b096a" candidate_uuid: "826e312a29ed44e286fbd752c626acfd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b99632ff7e84f99a05e3e8bd6435268" is_pre_election: true
I20260812 16:37:33.215306 14116 raft_consensus.cc:2454] T 325b1ff6466f4d1bbb4a5889a51b096a P 8b99632ff7e84f99a05e3e8bd6435268 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 826e312a29ed44e286fbd752c626acfd in term 0.
I20260812 16:37:33.215286 14336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "325b1ff6466f4d1bbb4a5889a51b096a" candidate_uuid: "826e312a29ed44e286fbd752c626acfd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0af3a5dcce9d4ae890aecc4f9f4ff6f7" is_pre_election: true
I20260812 16:37:33.215435 14336 raft_consensus.cc:2454] T 325b1ff6466f4d1bbb4a5889a51b096a P 0af3a5dcce9d4ae890aecc4f9f4ff6f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 826e312a29ed44e286fbd752c626acfd in term 0.
I20260812 16:37:33.215574 13790 leader_election.cc:302] T 325b1ff6466f4d1bbb4a5889a51b096a P 826e312a29ed44e286fbd752c626acfd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 826e312a29ed44e286fbd752c626acfd, 8b99632ff7e84f99a05e3e8bd6435268; no voters:
I20260812 16:37:33.215718 14483 raft_consensus.cc:2781] T 325b1ff6466f4d1bbb4a5889a51b096a P 826e312a29ed44e286fbd752c626acfd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:37:33.215781 14483 raft_consensus.cc:482] T 325b1ff6466f4d1bbb4a5889a51b096a P 826e312a29ed44e286fbd752c626acfd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:37:33.215798 14483 raft_consensus.cc:3037] T 325b1ff6466f4d1bbb4a5889a51b096a P 826e312a29ed44e286fbd752c626acfd [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:33.216522 14483 raft_consensus.cc:504] T 325b1ff6466f4d1bbb4a5889a51b096a P 826e312a29ed44e286fbd752c626acfd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b99632ff7e84f99a05e3e8bd6435268" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 41387 } } peers { permanent_uuid: "0af3a5dcce9d4ae890aecc4f9f4ff6f7" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43725 } } peers { permanent_uuid: "826e312a29ed44e286fbd752c626acfd" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 36583 } }
I20260812 16:37:33.216665 14483 leader_election.cc:288] T 325b1ff6466f4d1bbb4a5889a51b096a P 826e312a29ed44e286fbd752c626acfd [CANDIDATE]: Term 1 election: Requested vote from peers 8b99632ff7e84f99a05e3e8bd6435268 (127.0.253.194:41387), 0af3a5dcce9d4ae890aecc4f9f4ff6f7 (127.0.253.195:43725)
I20260812 16:37:33.216821 14116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "325b1ff6466f4d1bbb4a5889a51b096a" candidate_uuid: "826e312a29ed44e286fbd752c626acfd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b99632ff7e84f99a05e3e8bd6435268"
I20260812 16:37:33.216831 14336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "325b1ff6466f4d1bbb4a5889a51b096a" candidate_uuid: "826e312a29ed44e286fbd752c626acfd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0af3a5dcce9d4ae890aecc4f9f4ff6f7"
I20260812 16:37:33.216907 14116 raft_consensus.cc:3037] T 325b1ff6466f4d1bbb4a5889a51b096a P 8b99632ff7e84f99a05e3e8bd6435268 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:33.216907 14336 raft_consensus.cc:3037] T 325b1ff6466f4d1bbb4a5889a51b096a P 0af3a5dcce9d4ae890aecc4f9f4ff6f7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:33.217804 14116 raft_consensus.cc:2454] T 325b1ff6466f4d1bbb4a5889a51b096a P 8b99632ff7e84f99a05e3e8bd6435268 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 826e312a29ed44e286fbd752c626acfd in term 1.
I20260812 16:37:33.217804 14336 raft_consensus.cc:2454] T 325b1ff6466f4d1bbb4a5889a51b096a P 0af3a5dcce9d4ae890aecc4f9f4ff6f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 826e312a29ed44e286fbd752c626acfd in term 1.
I20260812 16:37:33.217976 13790 leader_election.cc:302] T 325b1ff6466f4d1bbb4a5889a51b096a P 826e312a29ed44e286fbd752c626acfd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 826e312a29ed44e286fbd752c626acfd, 8b99632ff7e84f99a05e3e8bd6435268; no voters:
I20260812 16:37:33.218112 14483 raft_consensus.cc:2781] T 325b1ff6466f4d1bbb4a5889a51b096a P 826e312a29ed44e286fbd752c626acfd [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:37:33.218292 14483 raft_consensus.cc:686] T 325b1ff6466f4d1bbb4a5889a51b096a P 826e312a29ed44e286fbd752c626acfd [term 1 LEADER]: Becoming Leader. State: Replica: 826e312a29ed44e286fbd752c626acfd, State: Running, Role: LEADER
I20260812 16:37:33.218392 14483 consensus_queue.cc:237] T 325b1ff6466f4d1bbb4a5889a51b096a P 826e312a29ed44e286fbd752c626acfd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b99632ff7e84f99a05e3e8bd6435268" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 41387 } } peers { permanent_uuid: "0af3a5dcce9d4ae890aecc4f9f4ff6f7" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43725 } } peers { permanent_uuid: "826e312a29ed44e286fbd752c626acfd" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 36583 } }
I20260812 16:37:33.219269 12252 catalog_manager.cc:5495] T 325b1ff6466f4d1bbb4a5889a51b096a P 826e312a29ed44e286fbd752c626acfd reported cstate change: term changed from 0 to 1, leader changed from <none> to 826e312a29ed44e286fbd752c626acfd (127.0.253.193). New cstate: current_term: 1 leader_uuid: "826e312a29ed44e286fbd752c626acfd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b99632ff7e84f99a05e3e8bd6435268" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 41387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0af3a5dcce9d4ae890aecc4f9f4ff6f7" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "826e312a29ed44e286fbd752c626acfd" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 36583 } health_report { overall_health: HEALTHY } } }
I20260812 16:37:33.219884 13579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "283116b420234f088056a1f917a31bb7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: false } } peers { permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: NON_VOTER last_known_addr { host: "127.0.253.252" port: 41893 } attrs { promote: true } } }
I20260812 16:37:33.219995 13579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:33.220817 13579 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7: attempting to promote NON_VOTER 34e21dc2c7b040dcb9c0ad37f96e3b6e to VOTER
I20260812 16:37:33.220938 13579 consensus_queue.cc:237] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: false } } peers { permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 41893 } attrs { promote: false } }
I20260812 16:37:33.221396 13669 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [term 1 LEARNER]: Refusing update from remote peer 283116b420234f088056a1f917a31bb7: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260812 16:37:33.221396 13322 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [term 1 FOLLOWER]: Refusing update from remote peer 283116b420234f088056a1f917a31bb7: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260812 16:37:33.221593 14501 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260812 16:37:33.221692 13235 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 41893 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260812 16:37:33.222260 13579 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 34e21dc2c7b040dcb9c0ad37f96e3b6e (127.0.253.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: false } } peers { permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 41893 } attrs { promote: false } } }
I20260812 16:37:33.222441 13322 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 34e21dc2c7b040dcb9c0ad37f96e3b6e (127.0.253.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: false } } peers { permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 41893 } attrs { promote: false } } }
I20260812 16:37:33.222429 13669 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 34e21dc2c7b040dcb9c0ad37f96e3b6e (127.0.253.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: false } } peers { permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 41893 } attrs { promote: false } } }
I20260812 16:37:33.223155 13579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "283116b420234f088056a1f917a31bb7" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: false } } peers { permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 41893 } attrs { promote: false } } }
I20260812 16:37:33.223181 13235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "283116b420234f088056a1f917a31bb7" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: false } } peers { permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 41893 } attrs { promote: false } } }
I20260812 16:37:33.223233 13579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:33.223242 13235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:33.223377 14501 sys_catalog.cc:455] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "283116b420234f088056a1f917a31bb7" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: false } } peers { permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 41893 } attrs { promote: false } } }
I20260812 16:37:33.223464 14501 sys_catalog.cc:458] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7 [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:33.223423 13731 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "283116b420234f088056a1f917a31bb7" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: false } } peers { permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 41893 } attrs { promote: false } } }
I20260812 16:37:33.223515 13731 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:37:33.223649 14503 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "283116b420234f088056a1f917a31bb7" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: false } } peers { permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 41893 } attrs { promote: false } } }
I20260812 16:37:33.223743 14503 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:37:33.241909 14467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:33.239312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36864 (local address 127.0.253.193:36583)
0812 16:37:33.239384 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:33.239387 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:33.239413 (+ 26us) server_negotiation.cc:407] Connection header received
0812 16:37:33.239458 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:33.239459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:33.239481 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:33.239569 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:37:33.239745 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.240244 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.241018 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.241250 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.241328 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:33.241501 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:37:33.241626 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:37:33.241720 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:33.241798 (+ 78us) server_negotiation.cc:299] Negotiation successful
0812 16:37:33.241821 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 16:37:33.243989 14116 raft_consensus.cc:1261] T 325b1ff6466f4d1bbb4a5889a51b096a P 8b99632ff7e84f99a05e3e8bd6435268 [term 1 FOLLOWER]: Refusing update from remote peer 826e312a29ed44e286fbd752c626acfd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:37:33.244004 14336 raft_consensus.cc:1261] T 325b1ff6466f4d1bbb4a5889a51b096a P 0af3a5dcce9d4ae890aecc4f9f4ff6f7 [term 1 FOLLOWER]: Refusing update from remote peer 826e312a29ed44e286fbd752c626acfd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:37:33.244344 14496 consensus_queue.cc:1035] T 325b1ff6466f4d1bbb4a5889a51b096a P 826e312a29ed44e286fbd752c626acfd [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b99632ff7e84f99a05e3e8bd6435268" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 41387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:37:33.244483 14483 consensus_queue.cc:1035] T 325b1ff6466f4d1bbb4a5889a51b096a P 826e312a29ed44e286fbd752c626acfd [LEADER]: Connected to new peer: Peer: permanent_uuid: "0af3a5dcce9d4ae890aecc4f9f4ff6f7" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:37:33.247104 14509 mvcc.cc:205] Tried to move back new op lower bound from 7317719667684093952 to 7317719667583160320. Current Snapshot: MvccSnapshot[applied={T|T < 7317719667684093952}]
I20260812 16:37:33.247404 14510 mvcc.cc:205] Tried to move back new op lower bound from 7317719667684093952 to 7317719667583160320. Current Snapshot: MvccSnapshot[applied={T|T < 7317719667684093952}]
I20260812 16:37:33.249192 14511 mvcc.cc:205] Tried to move back new op lower bound from 7317719667684093952 to 7317719667583160320. Current Snapshot: MvccSnapshot[applied={T|T < 7317719667684093952}]
I20260812 16:37:33.250584 12250 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:37:33.250669 12250 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:37:33.252478 12250 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 283116b420234f088056a1f917a31bb7: Sending DeleteTablet for 3 replicas of tablet 325b1ff6466f4d1bbb4a5889a51b096a
I20260812 16:37:33.252813 14083 tablet_service.cc:1514] Processing DeleteTablet for tablet 325b1ff6466f4d1bbb4a5889a51b096a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:33 UTC) from {username='oryx', principal='oryx/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:46318
I20260812 16:37:33.252832 14299 tablet_service.cc:1514] Processing DeleteTablet for tablet 325b1ff6466f4d1bbb4a5889a51b096a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:33 UTC) from {username='oryx', principal='oryx/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:44418
I20260812 16:37:33.252903 13864 tablet_service.cc:1514] Processing DeleteTablet for tablet 325b1ff6466f4d1bbb4a5889a51b096a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:33 UTC) from {username='oryx', principal='oryx/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:36850
I20260812 16:37:33.253108 14527 tablet_replica.cc:330] T 325b1ff6466f4d1bbb4a5889a51b096a P 826e312a29ed44e286fbd752c626acfd: stopping tablet replica
I20260812 16:37:33.253190 14525 tablet_replica.cc:330] T 325b1ff6466f4d1bbb4a5889a51b096a P 0af3a5dcce9d4ae890aecc4f9f4ff6f7: stopping tablet replica
I20260812 16:37:33.253211 14527 raft_consensus.cc:2229] T 325b1ff6466f4d1bbb4a5889a51b096a P 826e312a29ed44e286fbd752c626acfd [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:37:33.253271 14525 raft_consensus.cc:2229] T 325b1ff6466f4d1bbb4a5889a51b096a P 0af3a5dcce9d4ae890aecc4f9f4ff6f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:37:33.253304 14527 raft_consensus.cc:2258] T 325b1ff6466f4d1bbb4a5889a51b096a P 826e312a29ed44e286fbd752c626acfd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:33.253363 14525 raft_consensus.cc:2258] T 325b1ff6466f4d1bbb4a5889a51b096a P 0af3a5dcce9d4ae890aecc4f9f4ff6f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:33.253861 14525 ts_tablet_manager.cc:1905] T 325b1ff6466f4d1bbb4a5889a51b096a P 0af3a5dcce9d4ae890aecc4f9f4ff6f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:37:33.253861 14527 ts_tablet_manager.cc:1905] T 325b1ff6466f4d1bbb4a5889a51b096a P 826e312a29ed44e286fbd752c626acfd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:37:33.254550 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 12202
I20260812 16:37:33.255564 14524 tablet_replica.cc:330] T 325b1ff6466f4d1bbb4a5889a51b096a P 8b99632ff7e84f99a05e3e8bd6435268: stopping tablet replica
I20260812 16:37:33.255601 14527 ts_tablet_manager.cc:1918] T 325b1ff6466f4d1bbb4a5889a51b096a P 826e312a29ed44e286fbd752c626acfd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:37:33.255663 14527 log.cc:1196] T 325b1ff6466f4d1bbb4a5889a51b096a P 826e312a29ed44e286fbd752c626acfd: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-0/wal/wals/325b1ff6466f4d1bbb4a5889a51b096a
I20260812 16:37:33.255668 14524 raft_consensus.cc:2229] T 325b1ff6466f4d1bbb4a5889a51b096a P 8b99632ff7e84f99a05e3e8bd6435268 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:37:33.255772 14524 raft_consensus.cc:2258] T 325b1ff6466f4d1bbb4a5889a51b096a P 8b99632ff7e84f99a05e3e8bd6435268 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:33.255934 14525 ts_tablet_manager.cc:1918] T 325b1ff6466f4d1bbb4a5889a51b096a P 0af3a5dcce9d4ae890aecc4f9f4ff6f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:37:33.255990 14525 log.cc:1196] T 325b1ff6466f4d1bbb4a5889a51b096a P 0af3a5dcce9d4ae890aecc4f9f4ff6f7: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-2/wal/wals/325b1ff6466f4d1bbb4a5889a51b096a
I20260812 16:37:33.256207 14525 ts_tablet_manager.cc:1939] T 325b1ff6466f4d1bbb4a5889a51b096a P 0af3a5dcce9d4ae890aecc4f9f4ff6f7: Deleting consensus metadata
I20260812 16:37:33.257810 14527 ts_tablet_manager.cc:1939] T 325b1ff6466f4d1bbb4a5889a51b096a P 826e312a29ed44e286fbd752c626acfd: Deleting consensus metadata
I20260812 16:37:33.257915 14524 ts_tablet_manager.cc:1905] T 325b1ff6466f4d1bbb4a5889a51b096a P 8b99632ff7e84f99a05e3e8bd6435268: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:37:33.262423 14524 ts_tablet_manager.cc:1918] T 325b1ff6466f4d1bbb4a5889a51b096a P 8b99632ff7e84f99a05e3e8bd6435268: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:37:33.262508 14524 log.cc:1196] T 325b1ff6466f4d1bbb4a5889a51b096a P 8b99632ff7e84f99a05e3e8bd6435268: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-1/wal/wals/325b1ff6466f4d1bbb4a5889a51b096a
I20260812 16:37:33.262809 14524 ts_tablet_manager.cc:1939] T 325b1ff6466f4d1bbb4a5889a51b096a P 8b99632ff7e84f99a05e3e8bd6435268: Deleting consensus metadata
I20260812 16:37:33.268975 13978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:33.253855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35144 (local address 127.0.253.252:41893)
0812 16:37:33.253897 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:33.253899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:33.256645 (+ 2746us) server_negotiation.cc:407] Connection header received
0812 16:37:33.259267 (+ 2622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:33.259269 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:33.259281 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:33.259309 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:33.260060 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.261033 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.263241 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.263590 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.263637 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:33.265501 (+ 1864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:33.265503 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:33.265503 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:33.265512 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:33.267577 (+ 2065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:33.268005 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:33.268006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:33.268006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:33.268075 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:33.268314 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:33.268316 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:33.268316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:33.268485 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:33.268600 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:33.268740 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 16:37:33.268833 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 16:37:33.270612 13791 connection.cc:535] server connection from 127.0.0.1:36850 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 16:37:33.270946 14225 connection.cc:535] server connection from 127.0.0.1:44418 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 16:37:33.270961 14001 connection.cc:535] server connection from 127.0.0.1:46318 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 16:37:33.281352 13397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:33.272776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38776 (local address 127.0.253.253:43719)
0812 16:37:33.272829 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:33.272831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:33.273116 (+ 285us) server_negotiation.cc:407] Connection header received
0812 16:37:33.273223 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:33.273224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:33.273236 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:33.273332 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:33.274275 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.275014 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.275864 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.276180 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.276259 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:33.278214 (+ 1955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:33.278216 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:33.278216 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:33.278224 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:33.279931 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:33.280393 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:33.280395 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:33.280395 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:33.280463 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:33.280684 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:33.280685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:33.280686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:33.280876 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:33.280978 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:33.281142 (+ 164us) server_negotiation.cc:299] Negotiation successful
0812 16:37:33.281234 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 16:37:33.281423 13978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:33.272786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35150 (local address 127.0.253.252:41893)
0812 16:37:33.272825 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:33.272827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:33.273099 (+ 272us) server_negotiation.cc:407] Connection header received
0812 16:37:33.273200 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:33.273201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:33.273212 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:33.273241 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:33.274062 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.274858 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.276068 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:33.276375 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:33.276418 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:33.278190 (+ 1772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:33.278192 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:33.278192 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:33.278199 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:33.279941 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:33.280436 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:33.280437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:33.280437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:33.280491 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:33.280743 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:33.280744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:33.280744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:33.280884 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:33.280958 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:33.281245 (+ 287us) server_negotiation.cc:299] Negotiation successful
0812 16:37:33.281323 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 16:37:34.248953 14547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:34.248649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.253.195:37173)
0812 16:37:34.248821 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:34.248904 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":112,"threads_started":1}
W20260812 16:37:34.249359 14421 heartbeater.cc:644] Failed to heartbeat to 127.0.253.254:39023 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
I20260812 16:37:34.249890 14548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:34.249659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.253.193:55891)
0812 16:37:34.249766 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:34.249845 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":62,"threads_started":1}
W20260812 16:37:34.250110 13968 heartbeater.cc:644] Failed to heartbeat to 127.0.253.254:39023 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
I20260812 16:37:34.250993 14550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:34.250767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.253.194:38131)
0812 16:37:34.250894 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:34.250956 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":67,"threads_started":1}
W20260812 16:37:34.251199 14190 heartbeater.cc:644] Failed to heartbeat to 127.0.253.254:39023 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
I20260812 16:37:34.752560 14553 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 283116b420234f088056a1f917a31bb7)
I20260812 16:37:34.752736 14557 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 283116b420234f088056a1f917a31bb7)
I20260812 16:37:34.752758 14553 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: false } } peers { permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 41893 } attrs { promote: false } }
I20260812 16:37:34.752880 14557 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: false } } peers { permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 41893 } attrs { promote: false } }
I20260812 16:37:34.753125 14553 leader_election.cc:288] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 283116b420234f088056a1f917a31bb7 (127.0.253.254:39023), 9f12b08685a648f6ad5f1cca4c111d97 (127.0.253.253:43719)
I20260812 16:37:34.753473 14560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:34.753253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.0.1:54770)
0812 16:37:34.753401 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:34.753438 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":82,"threads_started":1}
I20260812 16:37:34.753473 14558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:34.753253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.0.1:54776)
0812 16:37:34.753372 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:34.753434 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":52,"threads_started":1}
W20260812 16:37:34.753600 13628 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
W20260812 16:37:34.753729 13282 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
I20260812 16:37:34.753924 14557 leader_election.cc:288] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 283116b420234f088056a1f917a31bb7 (127.0.253.254:39023), 34e21dc2c7b040dcb9c0ad37f96e3b6e (127.0.253.252:41893)
I20260812 16:37:34.754088 14560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:34.753925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.0.1:54786)
0812 16:37:34.754035 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:34.754061 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":69,"threads_started":1}
W20260812 16:37:34.754172 13628 leader_election.cc:334] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 283116b420234f088056a1f917a31bb7 (127.0.253.254:39023): Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
I20260812 16:37:34.754236 14567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:34.754123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.0.1:54788)
0812 16:37:34.754161 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:34.754192 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260812 16:37:34.754320 13282 leader_election.cc:334] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 283116b420234f088056a1f917a31bb7 (127.0.253.254:39023): Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
I20260812 16:37:34.756564 14558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:34.753509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:41893 (local address 127.0.0.1:35152)
0812 16:37:34.753556 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:34.753566 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:34.753630 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:34.754059 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:34.754060 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:34.754063 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:34.754133 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:34.754136 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:34.754872 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:34.754873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:34.755699 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:34.755702 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:34.756302 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:34.756303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:34.756364 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:34.756368 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:37:34.756444 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 16:37:34.756483 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"thread_start_us":60,"threads_started":1}
I20260812 16:37:34.756850 14563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:34.753293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.253:43719 (local address 127.0.0.1:38790)
0812 16:37:34.753512 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:34.753714 (+ 202us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:34.753780 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:34.754262 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:34.754263 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:34.754266 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:34.754324 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:34.754326 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:34.755050 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:34.755051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:34.755961 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:34.755962 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:34.756577 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:34.756578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:34.756652 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:34.756657 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:37:34.756725 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 16:37:34.756789 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":42,"threads_started":1}
I20260812 16:37:34.756868 14570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:34.753755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35152 (local address 127.0.253.252:41893)
0812 16:37:34.753886 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:34.753888 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:34.753896 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:37:34.753923 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:34.753924 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:34.753936 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:34.754204 (+ 268us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:34.754243 (+ 39us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:34.754793 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:34.755784 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:34.756233 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:34.756285 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:34.756294 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:34.756454 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 16:37:34.756566 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":78,"threads_started":1}
I20260812 16:37:34.757098 14565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:34.753339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38790 (local address 127.0.253.253:43719)
0812 16:37:34.753463 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:34.753465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:34.753858 (+ 393us) server_negotiation.cc:407] Connection header received
0812 16:37:34.753956 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:34.753957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:34.754023 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:34.754092 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:34.754414 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:34.754965 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:34.756042 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:34.756475 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:34.756532 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:34.756541 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:34.756962 (+ 421us) server_negotiation.cc:299] Negotiation successful
0812 16:37:34.757017 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":31,"threads_started":1}
I20260812 16:37:34.757217 13669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9f12b08685a648f6ad5f1cca4c111d97" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" is_pre_election: true
I20260812 16:37:34.757339 13669 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f12b08685a648f6ad5f1cca4c111d97 in term 1.
I20260812 16:37:34.757342 13322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9f12b08685a648f6ad5f1cca4c111d97" is_pre_election: true
I20260812 16:37:34.757444 13322 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34e21dc2c7b040dcb9c0ad37f96e3b6e in term 1.
I20260812 16:37:34.757516 13281 leader_election.cc:302] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34e21dc2c7b040dcb9c0ad37f96e3b6e, 9f12b08685a648f6ad5f1cca4c111d97; no voters: 283116b420234f088056a1f917a31bb7
I20260812 16:37:34.757617 14557 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 16:37:34.757651 14557 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 283116b420234f088056a1f917a31bb7)
I20260812 16:37:34.757676 14557 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [term 1 FOLLOWER]: Advancing to term 2
I20260812 16:37:34.757709 13625 leader_election.cc:302] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34e21dc2c7b040dcb9c0ad37f96e3b6e, 9f12b08685a648f6ad5f1cca4c111d97; no voters: 283116b420234f088056a1f917a31bb7
I20260812 16:37:34.758461 14557 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: false } } peers { permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 41893 } attrs { promote: false } }
I20260812 16:37:34.758576 14557 leader_election.cc:288] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [CANDIDATE]: Term 2 election: Requested vote from peers 283116b420234f088056a1f917a31bb7 (127.0.253.254:39023), 34e21dc2c7b040dcb9c0ad37f96e3b6e (127.0.253.252:41893)
I20260812 16:37:34.766662 14558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:34.766558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.0.1:54812)
0812 16:37:34.766600 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:34.766631 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260812 16:37:34.767129 14553 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 16:37:34.767190 14553 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 283116b420234f088056a1f917a31bb7)
I20260812 16:37:34.767187 14558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:34.767084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.0.1:54820)
0812 16:37:34.767140 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:34.767162 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 16:37:34.767212 14553 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [term 1 FOLLOWER]: Advancing to term 2
W20260812 16:37:34.767271 13282 leader_election.cc:334] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 283116b420234f088056a1f917a31bb7 (127.0.253.254:39023): Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
I20260812 16:37:34.767170 13669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9f12b08685a648f6ad5f1cca4c111d97" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e"
I20260812 16:37:34.767995 14553 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: false } } peers { permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 41893 } attrs { promote: false } }
I20260812 16:37:34.768148 13669 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9f12b08685a648f6ad5f1cca4c111d97 in current term 2: Already voted for candidate 34e21dc2c7b040dcb9c0ad37f96e3b6e in this term.
I20260812 16:37:34.768172 14553 leader_election.cc:288] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [CANDIDATE]: Term 2 election: Requested vote from peers 283116b420234f088056a1f917a31bb7 (127.0.253.254:39023), 9f12b08685a648f6ad5f1cca4c111d97 (127.0.253.253:43719)
I20260812 16:37:34.768317 13322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9f12b08685a648f6ad5f1cca4c111d97"
I20260812 16:37:34.768394 13281 leader_election.cc:302] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9f12b08685a648f6ad5f1cca4c111d97; no voters: 283116b420234f088056a1f917a31bb7, 34e21dc2c7b040dcb9c0ad37f96e3b6e
I20260812 16:37:34.768405 13322 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 34e21dc2c7b040dcb9c0ad37f96e3b6e in current term 2: Already voted for candidate 9f12b08685a648f6ad5f1cca4c111d97 in this term.
I20260812 16:37:34.768522 14557 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260812 16:37:34.768836 14563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:34.768250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.0.1:54836)
0812 16:37:34.768773 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:34.768803 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":506}
I20260812 16:37:34.769145 14563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:34.769059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.0.1:54848)
0812 16:37:34.769101 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:34.769124 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260812 16:37:34.769224 13628 leader_election.cc:334] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 283116b420234f088056a1f917a31bb7 (127.0.253.254:39023): Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
I20260812 16:37:34.769266 13628 leader_election.cc:302] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34e21dc2c7b040dcb9c0ad37f96e3b6e; no voters: 283116b420234f088056a1f917a31bb7, 9f12b08685a648f6ad5f1cca4c111d97
I20260812 16:37:34.769343 14553 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260812 16:37:35.250106 14575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:35.249819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.253.195:33679)
0812 16:37:35.250010 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:35.250059 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":91,"threads_started":1}
I20260812 16:37:35.250912 14576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:35.250415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.253.193:41445)
0812 16:37:35.250678 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:35.250792 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":90,"threads_started":1}
I20260812 16:37:35.251629 14578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:35.251443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.253.194:55793)
0812 16:37:35.251552 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:35.251584 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":67,"threads_started":1}
I20260812 16:37:36.250959 14583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:36.250655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.253.195:50477)
0812 16:37:36.250849 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:36.250898 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":87,"threads_started":1}
I20260812 16:37:36.251813 14584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:36.251399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.253.193:58113)
0812 16:37:36.251717 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:36.251759 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":57,"threads_started":1}
I20260812 16:37:36.252266 14585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:36.252020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.253.194:50083)
0812 16:37:36.252187 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:36.252232 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":88,"threads_started":1}
I20260812 16:37:36.301347 14586 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:37:36.301441 14586 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: false } } peers { permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 41893 } attrs { promote: false } }
I20260812 16:37:36.301612 14586 leader_election.cc:288] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 283116b420234f088056a1f917a31bb7 (127.0.253.254:39023), 9f12b08685a648f6ad5f1cca4c111d97 (127.0.253.253:43719)
I20260812 16:37:36.301954 13322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9f12b08685a648f6ad5f1cca4c111d97" is_pre_election: true
I20260812 16:37:36.302031 14588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:36.301790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.0.1:50824)
0812 16:37:36.301938 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:36.301991 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":72,"threads_started":1}
I20260812 16:37:36.302075 13322 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34e21dc2c7b040dcb9c0ad37f96e3b6e in term 2.
I20260812 16:37:36.302285 13625 leader_election.cc:302] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34e21dc2c7b040dcb9c0ad37f96e3b6e, 9f12b08685a648f6ad5f1cca4c111d97; no voters:
I20260812 16:37:36.302400 14586 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 16:37:36.302446 14586 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:37:36.302470 14586 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [term 2 FOLLOWER]: Advancing to term 3
I20260812 16:37:36.302488 14588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:36.302390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.0.1:50840)
0812 16:37:36.302426 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:36.302459 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260812 16:37:36.302654 13628 leader_election.cc:334] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 283116b420234f088056a1f917a31bb7 (127.0.253.254:39023): Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
I20260812 16:37:36.303369 14586 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: false } } peers { permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 41893 } attrs { promote: false } }
I20260812 16:37:36.303514 14586 leader_election.cc:288] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [CANDIDATE]: Term 3 election: Requested vote from peers 283116b420234f088056a1f917a31bb7 (127.0.253.254:39023), 9f12b08685a648f6ad5f1cca4c111d97 (127.0.253.253:43719)
I20260812 16:37:36.303707 14588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:36.303577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.0.1:50842)
0812 16:37:36.303653 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:36.303683 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260812 16:37:36.303673 13322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9f12b08685a648f6ad5f1cca4c111d97"
I20260812 16:37:36.303750 13322 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [term 2 FOLLOWER]: Advancing to term 3
I20260812 16:37:36.304044 14588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:36.303952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.0.1:50856)
0812 16:37:36.303998 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:36.304025 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 16:37:36.304164 13628 leader_election.cc:334] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 283116b420234f088056a1f917a31bb7 (127.0.253.254:39023): Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
I20260812 16:37:36.304548 13322 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34e21dc2c7b040dcb9c0ad37f96e3b6e in term 3.
I20260812 16:37:36.304739 13625 leader_election.cc:302] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34e21dc2c7b040dcb9c0ad37f96e3b6e, 9f12b08685a648f6ad5f1cca4c111d97; no voters: 283116b420234f088056a1f917a31bb7
I20260812 16:37:36.304836 14586 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [term 3 FOLLOWER]: Leader election won for term 3
I20260812 16:37:36.304934 14586 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [term 3 LEADER]: Becoming Leader. State: Replica: 34e21dc2c7b040dcb9c0ad37f96e3b6e, State: Running, Role: LEADER
I20260812 16:37:36.305025 14586 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: false } } peers { permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 41893 } attrs { promote: false } }
I20260812 16:37:36.305408 14590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34e21dc2c7b040dcb9c0ad37f96e3b6e. Latest consensus state: current_term: 3 leader_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: false } } peers { permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 41893 } attrs { promote: false } } }
I20260812 16:37:36.305531 14590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:36.305742 14597 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:37:36.306468 14597 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:37:36.306573 14597 catalog_manager.cc:1259] Loaded cluster ID: 9d5f31f6600341f5982998fd8c1db852
I20260812 16:37:36.306607 14597 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:37:36.306795 14597 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:37:36.306890 14597 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e: Loaded TSK: 0
I20260812 16:37:36.307037 14597 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:37:36.370198 13652 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35150:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:37:36.372131 13652 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:37:36.373371 14588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:36.373205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.0.1:50886)
0812 16:37:36.373291 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:36.373332 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260812 16:37:36.373469 13322 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [term 3 FOLLOWER]: Refusing update from remote peer 34e21dc2c7b040dcb9c0ad37f96e3b6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260812 16:37:36.373790 14588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:36.373674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.0.1:50890)
0812 16:37:36.373732 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:36.373764 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 16:37:36.373814 14590 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260812 16:37:36.373950 13628 consensus_peers.cc:487] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e -> Peer 283116b420234f088056a1f917a31bb7 (127.0.253.254:39023): Couldn't send request to peer 283116b420234f088056a1f917a31bb7. Status: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 16:37:36.374549 14599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34e21dc2c7b040dcb9c0ad37f96e3b6e. Latest consensus state: current_term: 3 leader_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: false } } peers { permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 41893 } attrs { promote: false } } }
I20260812 16:37:36.374666 14599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:37:36.375185 14590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: false } } peers { permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 41893 } attrs { promote: false } } }
I20260812 16:37:36.375293 14590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:36.375254 14599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "283116b420234f088056a1f917a31bb7" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39023 } } peers { permanent_uuid: "9f12b08685a648f6ad5f1cca4c111d97" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43719 } attrs { promote: false } } peers { permanent_uuid: "34e21dc2c7b040dcb9c0ad37f96e3b6e" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 41893 } attrs { promote: false } } }
I20260812 16:37:36.375345 14599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f12b08685a648f6ad5f1cca4c111d97 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:37:36.380716 14607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:36.377229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:41387 (local address 127.0.0.1:34260)
0812 16:37:36.377638 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:36.377650 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:36.377712 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:36.377913 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:36.377914 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:36.377916 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:36.377991 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:36.377993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:36.378995 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:36.378996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:36.379841 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:36.379844 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:36.380474 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:36.380476 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:36.380539 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:36.380544 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:37:36.380594 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 16:37:36.380634 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"mutex_wait_us":35,"thread_start_us":114,"threads_started":1}
I20260812 16:37:36.380997 14611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:36.377322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34260 (local address 127.0.253.194:41387)
0812 16:37:36.377448 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:36.377451 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:36.377701 (+ 250us) server_negotiation.cc:407] Connection header received
0812 16:37:36.377770 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:36.377771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:36.377784 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:36.377855 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:36.378088 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:36.378894 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:36.379945 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:36.380341 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:36.380426 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:36.380439 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:36.380871 (+ 432us) server_negotiation.cc:299] Negotiation successful
0812 16:37:36.380927 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":67,"threads_started":1}
I20260812 16:37:36.381207 14588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:36.377146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.195:43725 (local address 127.0.0.1:57714)
0812 16:37:36.377604 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:36.377615 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:36.377678 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:36.378054 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:36.378055 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:36.378056 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:36.378104 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:36.378105 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:36.378754 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:36.378755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:36.379846 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:36.379849 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:36.380965 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:36.380966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:36.381032 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:36.381037 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:37:36.381088 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 16:37:36.381128 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442}
I20260812 16:37:36.381322 14608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:36.377239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:36583 (local address 127.0.0.1:60358)
0812 16:37:36.377884 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:36.377894 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:36.377950 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:36.378137 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:36.378138 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:36.378140 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:36.378176 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:36.378177 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:36.379023 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:36.379024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:36.380323 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:36.380326 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:36.381069 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:36.381070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:36.381124 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:36.381126 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 16:37:36.381174 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 16:37:36.381215 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"mutex_wait_us":32,"thread_start_us":126,"threads_started":1}
I20260812 16:37:36.381325 14616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:36.377489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60358 (local address 127.0.253.193:36583)
0812 16:37:36.377818 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:36.377820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:36.377958 (+ 138us) server_negotiation.cc:407] Connection header received
0812 16:37:36.377995 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:36.377996 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:36.378008 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:36.378090 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:36.378247 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:36.378945 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:36.380412 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:36.380990 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:36.381073 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:36.381090 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:36.381213 (+ 123us) server_negotiation.cc:299] Negotiation successful
0812 16:37:36.381261 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":54,"threads_started":1}
I20260812 16:37:36.381388 14613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:36.377468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57714 (local address 127.0.253.195:43725)
0812 16:37:36.377599 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:36.377600 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:36.377929 (+ 329us) server_negotiation.cc:407] Connection header received
0812 16:37:36.377962 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:36.377963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:36.377975 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:36.378053 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:36.378162 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:36.378658 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:36.380408 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:36.380845 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:36.380907 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:36.380918 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:36.381250 (+ 332us) server_negotiation.cc:299] Negotiation successful
0812 16:37:36.381303 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":86,"threads_started":1}
I20260812 16:37:36.381763 13864 tablet_service.cc:1467] Processing CreateTablet for tablet 3ba8c61247c3496d99c0b299594919a7 (DEFAULT_TABLE table=test-table [id=023f19585b8e4e99ba8606021c29941b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:36.381821 14299 tablet_service.cc:1467] Processing CreateTablet for tablet 3ba8c61247c3496d99c0b299594919a7 (DEFAULT_TABLE table=test-table [id=023f19585b8e4e99ba8606021c29941b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:36.381826 14083 tablet_service.cc:1467] Processing CreateTablet for tablet 3ba8c61247c3496d99c0b299594919a7 (DEFAULT_TABLE table=test-table [id=023f19585b8e4e99ba8606021c29941b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:36.381930 13864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ba8c61247c3496d99c0b299594919a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:36.381934 14083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ba8c61247c3496d99c0b299594919a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:36.381950 14299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ba8c61247c3496d99c0b299594919a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:36.383376 14617 tablet_bootstrap.cc:492] T 3ba8c61247c3496d99c0b299594919a7 P 0af3a5dcce9d4ae890aecc4f9f4ff6f7: Bootstrap starting.
I20260812 16:37:36.383625 14621 tablet_bootstrap.cc:492] T 3ba8c61247c3496d99c0b299594919a7 P 8b99632ff7e84f99a05e3e8bd6435268: Bootstrap starting.
I20260812 16:37:36.383806 14617 tablet_bootstrap.cc:654] T 3ba8c61247c3496d99c0b299594919a7 P 0af3a5dcce9d4ae890aecc4f9f4ff6f7: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:36.383847 14622 tablet_bootstrap.cc:492] T 3ba8c61247c3496d99c0b299594919a7 P 826e312a29ed44e286fbd752c626acfd: Bootstrap starting.
I20260812 16:37:36.384027 14621 tablet_bootstrap.cc:654] T 3ba8c61247c3496d99c0b299594919a7 P 8b99632ff7e84f99a05e3e8bd6435268: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:36.384303 14622 tablet_bootstrap.cc:654] T 3ba8c61247c3496d99c0b299594919a7 P 826e312a29ed44e286fbd752c626acfd: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:36.384485 14617 tablet_bootstrap.cc:492] T 3ba8c61247c3496d99c0b299594919a7 P 0af3a5dcce9d4ae890aecc4f9f4ff6f7: No bootstrap required, opened a new log
I20260812 16:37:36.384542 14617 ts_tablet_manager.cc:1397] T 3ba8c61247c3496d99c0b299594919a7 P 0af3a5dcce9d4ae890aecc4f9f4ff6f7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:36.384608 14621 tablet_bootstrap.cc:492] T 3ba8c61247c3496d99c0b299594919a7 P 8b99632ff7e84f99a05e3e8bd6435268: No bootstrap required, opened a new log
I20260812 16:37:36.384672 14621 ts_tablet_manager.cc:1397] T 3ba8c61247c3496d99c0b299594919a7 P 8b99632ff7e84f99a05e3e8bd6435268: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:36.384714 14617 raft_consensus.cc:348] T 3ba8c61247c3496d99c0b299594919a7 P 0af3a5dcce9d4ae890aecc4f9f4ff6f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0af3a5dcce9d4ae890aecc4f9f4ff6f7" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43725 } } peers { permanent_uuid: "8b99632ff7e84f99a05e3e8bd6435268" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 41387 } } peers { permanent_uuid: "826e312a29ed44e286fbd752c626acfd" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 36583 } }
I20260812 16:37:36.384768 14617 raft_consensus.cc:374] T 3ba8c61247c3496d99c0b299594919a7 P 0af3a5dcce9d4ae890aecc4f9f4ff6f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:36.384783 14617 raft_consensus.cc:729] T 3ba8c61247c3496d99c0b299594919a7 P 0af3a5dcce9d4ae890aecc4f9f4ff6f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0af3a5dcce9d4ae890aecc4f9f4ff6f7, State: Initialized, Role: FOLLOWER
I20260812 16:37:36.384792 14621 raft_consensus.cc:348] T 3ba8c61247c3496d99c0b299594919a7 P 8b99632ff7e84f99a05e3e8bd6435268 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0af3a5dcce9d4ae890aecc4f9f4ff6f7" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43725 } } peers { permanent_uuid: "8b99632ff7e84f99a05e3e8bd6435268" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 41387 } } peers { permanent_uuid: "826e312a29ed44e286fbd752c626acfd" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 36583 } }
I20260812 16:37:36.384863 14621 raft_consensus.cc:374] T 3ba8c61247c3496d99c0b299594919a7 P 8b99632ff7e84f99a05e3e8bd6435268 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:36.384841 14617 consensus_queue.cc:260] T 3ba8c61247c3496d99c0b299594919a7 P 0af3a5dcce9d4ae890aecc4f9f4ff6f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0af3a5dcce9d4ae890aecc4f9f4ff6f7" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43725 } } peers { permanent_uuid: "8b99632ff7e84f99a05e3e8bd6435268" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 41387 } } peers { permanent_uuid: "826e312a29ed44e286fbd752c626acfd" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 36583 } }
I20260812 16:37:36.384891 14621 raft_consensus.cc:729] T 3ba8c61247c3496d99c0b299594919a7 P 8b99632ff7e84f99a05e3e8bd6435268 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b99632ff7e84f99a05e3e8bd6435268, State: Initialized, Role: FOLLOWER
I20260812 16:37:36.384948 14621 consensus_queue.cc:260] T 3ba8c61247c3496d99c0b299594919a7 P 8b99632ff7e84f99a05e3e8bd6435268 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0af3a5dcce9d4ae890aecc4f9f4ff6f7" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43725 } } peers { permanent_uuid: "8b99632ff7e84f99a05e3e8bd6435268" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 41387 } } peers { permanent_uuid: "826e312a29ed44e286fbd752c626acfd" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 36583 } }
I20260812 16:37:36.385062 14617 ts_tablet_manager.cc:1428] T 3ba8c61247c3496d99c0b299594919a7 P 0af3a5dcce9d4ae890aecc4f9f4ff6f7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:36.385233 14621 ts_tablet_manager.cc:1428] T 3ba8c61247c3496d99c0b299594919a7 P 8b99632ff7e84f99a05e3e8bd6435268: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:36.385335 14583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:36.385192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.253.195:44259)
0812 16:37:36.385281 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:36.385307 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260812 16:37:36.385802 14585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:36.385702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.253.194:41723)
0812 16:37:36.385743 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:36.385772 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260812 16:37:36.385815 14423 heartbeater.cc:503] Master 127.0.253.252:41893 was elected leader, sending a full tablet report...
I20260812 16:37:36.385867 14191 heartbeater.cc:503] Master 127.0.253.252:41893 was elected leader, sending a full tablet report...
I20260812 16:37:36.386372 14585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:36.386284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.253.194:55171)
0812 16:37:36.386324 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:36.386348 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260812 16:37:36.386613 14583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:36.386523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.253.195:33567)
0812 16:37:36.386564 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:36.386591 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260812 16:37:36.385021 14622 tablet_bootstrap.cc:492] T 3ba8c61247c3496d99c0b299594919a7 P 826e312a29ed44e286fbd752c626acfd: No bootstrap required, opened a new log
I20260812 16:37:36.386881 14622 ts_tablet_manager.cc:1397] T 3ba8c61247c3496d99c0b299594919a7 P 826e312a29ed44e286fbd752c626acfd: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 16:37:36.386998 14622 raft_consensus.cc:348] T 3ba8c61247c3496d99c0b299594919a7 P 826e312a29ed44e286fbd752c626acfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0af3a5dcce9d4ae890aecc4f9f4ff6f7" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43725 } } peers { permanent_uuid: "8b99632ff7e84f99a05e3e8bd6435268" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 41387 } } peers { permanent_uuid: "826e312a29ed44e286fbd752c626acfd" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 36583 } }
I20260812 16:37:36.387096 14622 raft_consensus.cc:374] T 3ba8c61247c3496d99c0b299594919a7 P 826e312a29ed44e286fbd752c626acfd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:36.387130 14622 raft_consensus.cc:729] T 3ba8c61247c3496d99c0b299594919a7 P 826e312a29ed44e286fbd752c626acfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 826e312a29ed44e286fbd752c626acfd, State: Initialized, Role: FOLLOWER
I20260812 16:37:36.387176 14622 consensus_queue.cc:260] T 3ba8c61247c3496d99c0b299594919a7 P 826e312a29ed44e286fbd752c626acfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0af3a5dcce9d4ae890aecc4f9f4ff6f7" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43725 } } peers { permanent_uuid: "8b99632ff7e84f99a05e3e8bd6435268" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 41387 } } peers { permanent_uuid: "826e312a29ed44e286fbd752c626acfd" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 36583 } }
I20260812 16:37:36.387351 14622 ts_tablet_manager.cc:1428] T 3ba8c61247c3496d99c0b299594919a7 P 826e312a29ed44e286fbd752c626acfd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:36.387619 14584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:36.387516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.253.193:54445)
0812 16:37:36.387559 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:36.387586 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260812 16:37:36.387840 13969 heartbeater.cc:503] Master 127.0.253.252:41893 was elected leader, sending a full tablet report...
I20260812 16:37:36.387902 14584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:36.387819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.253.193:44943)
0812 16:37:36.387854 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:36.387877 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260812 16:37:36.473615 14632 raft_consensus.cc:482] T 3ba8c61247c3496d99c0b299594919a7 P 826e312a29ed44e286fbd752c626acfd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:37:36.473724 14632 raft_consensus.cc:504] T 3ba8c61247c3496d99c0b299594919a7 P 826e312a29ed44e286fbd752c626acfd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0af3a5dcce9d4ae890aecc4f9f4ff6f7" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43725 } } peers { permanent_uuid: "8b99632ff7e84f99a05e3e8bd6435268" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 41387 } } peers { permanent_uuid: "826e312a29ed44e286fbd752c626acfd" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 36583 } }
I20260812 16:37:36.473935 14632 leader_election.cc:288] T 3ba8c61247c3496d99c0b299594919a7 P 826e312a29ed44e286fbd752c626acfd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0af3a5dcce9d4ae890aecc4f9f4ff6f7 (127.0.253.195:43725), 8b99632ff7e84f99a05e3e8bd6435268 (127.0.253.194:41387)
I20260812 16:37:36.474220 14116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ba8c61247c3496d99c0b299594919a7" candidate_uuid: "826e312a29ed44e286fbd752c626acfd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b99632ff7e84f99a05e3e8bd6435268" is_pre_election: true
I20260812 16:37:36.474220 14336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ba8c61247c3496d99c0b299594919a7" candidate_uuid: "826e312a29ed44e286fbd752c626acfd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0af3a5dcce9d4ae890aecc4f9f4ff6f7" is_pre_election: true
I20260812 16:37:36.474342 14116 raft_consensus.cc:2454] T 3ba8c61247c3496d99c0b299594919a7 P 8b99632ff7e84f99a05e3e8bd6435268 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 826e312a29ed44e286fbd752c626acfd in term 0.
I20260812 16:37:36.474345 14336 raft_consensus.cc:2454] T 3ba8c61247c3496d99c0b299594919a7 P 0af3a5dcce9d4ae890aecc4f9f4ff6f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 826e312a29ed44e286fbd752c626acfd in term 0.
I20260812 16:37:36.474557 13791 leader_election.cc:302] T 3ba8c61247c3496d99c0b299594919a7 P 826e312a29ed44e286fbd752c626acfd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0af3a5dcce9d4ae890aecc4f9f4ff6f7, 826e312a29ed44e286fbd752c626acfd; no voters:
I20260812 16:37:36.474701 14632 raft_consensus.cc:2781] T 3ba8c61247c3496d99c0b299594919a7 P 826e312a29ed44e286fbd752c626acfd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:37:36.474759 14632 raft_consensus.cc:482] T 3ba8c61247c3496d99c0b299594919a7 P 826e312a29ed44e286fbd752c626acfd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:37:36.474774 14632 raft_consensus.cc:3037] T 3ba8c61247c3496d99c0b299594919a7 P 826e312a29ed44e286fbd752c626acfd [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:36.475607 14632 raft_consensus.cc:504] T 3ba8c61247c3496d99c0b299594919a7 P 826e312a29ed44e286fbd752c626acfd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0af3a5dcce9d4ae890aecc4f9f4ff6f7" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43725 } } peers { permanent_uuid: "8b99632ff7e84f99a05e3e8bd6435268" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 41387 } } peers { permanent_uuid: "826e312a29ed44e286fbd752c626acfd" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 36583 } }
I20260812 16:37:36.475746 14632 leader_election.cc:288] T 3ba8c61247c3496d99c0b299594919a7 P 826e312a29ed44e286fbd752c626acfd [CANDIDATE]: Term 1 election: Requested vote from peers 0af3a5dcce9d4ae890aecc4f9f4ff6f7 (127.0.253.195:43725), 8b99632ff7e84f99a05e3e8bd6435268 (127.0.253.194:41387)
I20260812 16:37:36.475910 14336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ba8c61247c3496d99c0b299594919a7" candidate_uuid: "826e312a29ed44e286fbd752c626acfd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0af3a5dcce9d4ae890aecc4f9f4ff6f7"
I20260812 16:37:36.475910 14116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ba8c61247c3496d99c0b299594919a7" candidate_uuid: "826e312a29ed44e286fbd752c626acfd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b99632ff7e84f99a05e3e8bd6435268"
I20260812 16:37:36.475996 14336 raft_consensus.cc:3037] T 3ba8c61247c3496d99c0b299594919a7 P 0af3a5dcce9d4ae890aecc4f9f4ff6f7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:36.475996 14116 raft_consensus.cc:3037] T 3ba8c61247c3496d99c0b299594919a7 P 8b99632ff7e84f99a05e3e8bd6435268 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:36.476807 14116 raft_consensus.cc:2454] T 3ba8c61247c3496d99c0b299594919a7 P 8b99632ff7e84f99a05e3e8bd6435268 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 826e312a29ed44e286fbd752c626acfd in term 1.
I20260812 16:37:36.476807 14336 raft_consensus.cc:2454] T 3ba8c61247c3496d99c0b299594919a7 P 0af3a5dcce9d4ae890aecc4f9f4ff6f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 826e312a29ed44e286fbd752c626acfd in term 1.
I20260812 16:37:36.477015 13791 leader_election.cc:302] T 3ba8c61247c3496d99c0b299594919a7 P 826e312a29ed44e286fbd752c626acfd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0af3a5dcce9d4ae890aecc4f9f4ff6f7, 826e312a29ed44e286fbd752c626acfd; no voters:
I20260812 16:37:36.477120 14632 raft_consensus.cc:2781] T 3ba8c61247c3496d99c0b299594919a7 P 826e312a29ed44e286fbd752c626acfd [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:37:36.477329 14632 raft_consensus.cc:686] T 3ba8c61247c3496d99c0b299594919a7 P 826e312a29ed44e286fbd752c626acfd [term 1 LEADER]: Becoming Leader. State: Replica: 826e312a29ed44e286fbd752c626acfd, State: Running, Role: LEADER
I20260812 16:37:36.477404 14632 consensus_queue.cc:237] T 3ba8c61247c3496d99c0b299594919a7 P 826e312a29ed44e286fbd752c626acfd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0af3a5dcce9d4ae890aecc4f9f4ff6f7" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43725 } } peers { permanent_uuid: "8b99632ff7e84f99a05e3e8bd6435268" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 41387 } } peers { permanent_uuid: "826e312a29ed44e286fbd752c626acfd" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 36583 } }
I20260812 16:37:36.477702 14584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:36.477576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.253.193:40955)
0812 16:37:36.477631 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:36.477664 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 16:37:36.478188 13650 catalog_manager.cc:5495] T 3ba8c61247c3496d99c0b299594919a7 P 826e312a29ed44e286fbd752c626acfd reported cstate change: term changed from 0 to 1, leader changed from <none> to 826e312a29ed44e286fbd752c626acfd (127.0.253.193). New cstate: current_term: 1 leader_uuid: "826e312a29ed44e286fbd752c626acfd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0af3a5dcce9d4ae890aecc4f9f4ff6f7" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b99632ff7e84f99a05e3e8bd6435268" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 41387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "826e312a29ed44e286fbd752c626acfd" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 36583 } health_report { overall_health: HEALTHY } } }
I20260812 16:37:36.498003 14616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:36.495634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60370 (local address 127.0.253.193:36583)
0812 16:37:36.495701 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:36.495703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:36.495738 (+ 35us) server_negotiation.cc:407] Connection header received
0812 16:37:36.495819 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:36.495820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:36.495832 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:36.495900 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:37:36.496029 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:36.496514 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:36.497288 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:36.497487 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:36.497565 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:36.497728 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:37:36.497766 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:37:36.497825 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:36.497889 (+ 64us) server_negotiation.cc:299] Negotiation successful
0812 16:37:36.497913 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 16:37:36.499029 14116 raft_consensus.cc:1261] T 3ba8c61247c3496d99c0b299594919a7 P 8b99632ff7e84f99a05e3e8bd6435268 [term 1 FOLLOWER]: Refusing update from remote peer 826e312a29ed44e286fbd752c626acfd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:37:36.499116 14336 raft_consensus.cc:1261] T 3ba8c61247c3496d99c0b299594919a7 P 0af3a5dcce9d4ae890aecc4f9f4ff6f7 [term 1 FOLLOWER]: Refusing update from remote peer 826e312a29ed44e286fbd752c626acfd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:37:36.499394 14636 consensus_queue.cc:1035] T 3ba8c61247c3496d99c0b299594919a7 P 826e312a29ed44e286fbd752c626acfd [LEADER]: Connected to new peer: Peer: permanent_uuid: "0af3a5dcce9d4ae890aecc4f9f4ff6f7" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:37:36.499716 14632 consensus_queue.cc:1035] T 3ba8c61247c3496d99c0b299594919a7 P 826e312a29ed44e286fbd752c626acfd [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b99632ff7e84f99a05e3e8bd6435268" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 41387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:37:36.500192 14583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:36.500083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.253.195:44521)
0812 16:37:36.500128 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:36.500158 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260812 16:37:36.500859 14585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:36.500748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.253.194:45921)
0812 16:37:36.500793 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:36.500823 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260812 16:37:36.501719 14583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:36.501614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.253.195:40525)
0812 16:37:36.501659 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:36.501687 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260812 16:37:36.501873 14584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:36.501709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.253.193:49877)
0812 16:37:36.501811 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:36.501842 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260812 16:37:36.502734 14584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:36.502615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.253.193:44223)
0812 16:37:36.502674 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:36.502705 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 16:37:36.502795 14585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:36.502698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39023 (local address 127.0.253.194:45123)
0812 16:37:36.502735 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:36.502764 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260812 16:37:36.504762 13650 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:37:36.504858 13650 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:37:36.506120 13650 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 34e21dc2c7b040dcb9c0ad37f96e3b6e: Sending DeleteTablet for 3 replicas of tablet 3ba8c61247c3496d99c0b299594919a7
I20260812 16:37:36.506494 13864 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ba8c61247c3496d99c0b299594919a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:36 UTC) from {username='oryx', principal='oryx/127.0.253.252@KRBTEST.COM'} at 127.0.0.1:60358
I20260812 16:37:36.506590 14083 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ba8c61247c3496d99c0b299594919a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:36 UTC) from {username='oryx', principal='oryx/127.0.253.252@KRBTEST.COM'} at 127.0.0.1:34260
I20260812 16:37:36.506595 14299 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ba8c61247c3496d99c0b299594919a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:36 UTC) from {username='oryx', principal='oryx/127.0.253.252@KRBTEST.COM'} at 127.0.0.1:57714
I20260812 16:37:36.506804 14659 tablet_replica.cc:330] T 3ba8c61247c3496d99c0b299594919a7 P 8b99632ff7e84f99a05e3e8bd6435268: stopping tablet replica
I20260812 16:37:36.506821 14660 tablet_replica.cc:330] T 3ba8c61247c3496d99c0b299594919a7 P 0af3a5dcce9d4ae890aecc4f9f4ff6f7: stopping tablet replica
I20260812 16:37:36.506877 14660 raft_consensus.cc:2229] T 3ba8c61247c3496d99c0b299594919a7 P 0af3a5dcce9d4ae890aecc4f9f4ff6f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:37:36.506953 14660 raft_consensus.cc:2258] T 3ba8c61247c3496d99c0b299594919a7 P 0af3a5dcce9d4ae890aecc4f9f4ff6f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:36.506954 14657 tablet_replica.cc:330] T 3ba8c61247c3496d99c0b299594919a7 P 826e312a29ed44e286fbd752c626acfd: stopping tablet replica
I20260812 16:37:36.507016 14657 raft_consensus.cc:2229] T 3ba8c61247c3496d99c0b299594919a7 P 826e312a29ed44e286fbd752c626acfd [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:37:36.507274 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 13575
I20260812 16:37:36.507418 14660 ts_tablet_manager.cc:1905] T 3ba8c61247c3496d99c0b299594919a7 P 0af3a5dcce9d4ae890aecc4f9f4ff6f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:37:36.512773 14660 ts_tablet_manager.cc:1918] T 3ba8c61247c3496d99c0b299594919a7 P 0af3a5dcce9d4ae890aecc4f9f4ff6f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:37:36.512850 14660 log.cc:1196] T 3ba8c61247c3496d99c0b299594919a7 P 0af3a5dcce9d4ae890aecc4f9f4ff6f7: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-2/wal/wals/3ba8c61247c3496d99c0b299594919a7
I20260812 16:37:36.513115 14660 ts_tablet_manager.cc:1939] T 3ba8c61247c3496d99c0b299594919a7 P 0af3a5dcce9d4ae890aecc4f9f4ff6f7: Deleting consensus metadata
I20260812 16:37:36.532176 14659 raft_consensus.cc:2229] T 3ba8c61247c3496d99c0b299594919a7 P 8b99632ff7e84f99a05e3e8bd6435268 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:37:36.532321 14659 raft_consensus.cc:2258] T 3ba8c61247c3496d99c0b299594919a7 P 8b99632ff7e84f99a05e3e8bd6435268 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:36.534770 14659 ts_tablet_manager.cc:1905] T 3ba8c61247c3496d99c0b299594919a7 P 8b99632ff7e84f99a05e3e8bd6435268: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:37:36.537142 14659 ts_tablet_manager.cc:1918] T 3ba8c61247c3496d99c0b299594919a7 P 8b99632ff7e84f99a05e3e8bd6435268: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:37:36.537238 14659 log.cc:1196] T 3ba8c61247c3496d99c0b299594919a7 P 8b99632ff7e84f99a05e3e8bd6435268: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552619305671-1015-0/minicluster-data/ts-1/wal/wals/3ba8c61247c3496d99c0b299594919a7
I20260812 16:37:36.537526 14659 ts_tablet_manager.cc:1939] T 3ba8c61247c3496d99c0b299594919a7 P 8b99632ff7e84f99a05e3e8bd6435268: Deleting consensus metadata
I20260812 16:37:36.537523 13626 catalog_manager.cc:4841] TS 0af3a5dcce9d4ae890aecc4f9f4ff6f7 (127.0.253.195:43725): tablet 3ba8c61247c3496d99c0b299594919a7 (table test-table [id=023f19585b8e4e99ba8606021c29941b]) successfully deleted
I20260812 16:37:36.537917 13625 catalog_manager.cc:4841] TS 8b99632ff7e84f99a05e3e8bd6435268 (127.0.253.194:41387): tablet 3ba8c61247c3496d99c0b299594919a7 (table test-table [id=023f19585b8e4e99ba8606021c29941b]) successfully deleted
I20260812 16:37:36.543521 14608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:36.543375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:36583 (local address 127.0.0.1:60372)
0812 16:37:36.543444 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:36.543485 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.193:36583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 16:37:36.543629 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 13980
W20260812 16:37:36.543758 13626 catalog_manager.cc:4584] TS 826e312a29ed44e286fbd752c626acfd (127.0.253.193:36583): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3ba8c61247c3496d99c0b299594919a7: Network error: Client connection negotiation failed: client connection to 127.0.253.193:36583: connect: Connection refused (error 111)
I20260812 16:37:36.554579 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 14204
I20260812 16:37:36.565707 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 13077
I20260812 16:37:36.579417 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 13405
2026-08-12T16:37:36Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6707 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:37:36 dist-test-slave-w4v5 krb5kdc[14663](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:37:36 dist-test-slave-w4v5 krb5kdc[14663](info): set up 2 sockets
Aug 12 16:37:36 dist-test-slave-w4v5 krb5kdc[14663](info): commencing operation
krb5kdc: starting...
W20260812 16:37:37.911264 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.296s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:37:37 dist-test-slave-w4v5 krb5kdc[14663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:37:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:37:38Z Disabled control of system clock
WARNING: no policy specified for oryx/127.0.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.254.keytab.
Entry for principal oryx/127.0.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.254.keytab.
I20260812 16:37:38.042086 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:33909
--webserver_interface=127.0.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.254.keytab
--principal=oryx/127.0.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:41647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:33909
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:38.126653 14681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:38.126904 14681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:38.126940 14681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:38.128151 14681 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:37:38.128229 14681 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 16:37:38.128253 14681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:38.128278 14681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:38.128304 14681 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:37:38.128335 14681 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:37:38.129858 14681 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:41647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:33909
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.254.keytab
--principal=oryx/127.0.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:33909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:38.130151 14681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:38.130389 14681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:38.135434 14702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:38.135421 14696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:38.135449 14695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:38.135624 14681 server_base.cc:1034] running on GCE node
I20260812 16:37:38.135946 14681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:38.136283 14681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:38.137445 14681 hybrid_clock.cc:648] HybridClock initialized: now 1786552658137415 us; error 54 us; skew 500 ppm
Aug 12 16:37:38 dist-test-slave-w4v5 krb5kdc[14663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552658, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:38.140002 14681 init.cc:377] Logged in from keytab as oryx/127.0.253.254@KRBTEST.COM (short username oryx)
I20260812 16:37:38.140431 14681 webserver.cc:460] Webserver started at http://127.0.253.254:39313/ using document root <none> and password file <none>
I20260812 16:37:38.140652 14681 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:38.140698 14681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:38.140853 14681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:38.141898 14681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "d520c8ddc0ff4a1e9d90dfab76998e47"
format_stamp: "Formatted at 2026-08-12 16:37:38 on dist-test-slave-w4v5"
server_key: "ab75f17cbb4e8f65507d970f8692a3f7"
server_key_iv: "a7a77a03fc31d598dd1efa41ce17c8ac"
server_key_version: "encryptionkey@0"
I20260812 16:37:38.142294 14681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "d520c8ddc0ff4a1e9d90dfab76998e47"
format_stamp: "Formatted at 2026-08-12 16:37:38 on dist-test-slave-w4v5"
server_key: "ab75f17cbb4e8f65507d970f8692a3f7"
server_key_iv: "a7a77a03fc31d598dd1efa41ce17c8ac"
server_key_version: "encryptionkey@0"
I20260812 16:37:38.143803 14681 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 16:37:38.144634 14710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:38.144841 14681 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:38.144932 14681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "d520c8ddc0ff4a1e9d90dfab76998e47"
format_stamp: "Formatted at 2026-08-12 16:37:38 on dist-test-slave-w4v5"
server_key: "ab75f17cbb4e8f65507d970f8692a3f7"
server_key_iv: "a7a77a03fc31d598dd1efa41ce17c8ac"
server_key_version: "encryptionkey@0"
I20260812 16:37:38.144989 14681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:38.162935 14681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:38.165270 14681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:38.165385 14681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:38.170013 14681 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:33909
I20260812 16:37:38.170135 14831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:33909 every 8 connection(s)
I20260812 16:37:38.170459 14681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:37:38.171260 14832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:38.173628 14832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d520c8ddc0ff4a1e9d90dfab76998e47: Bootstrap starting.
I20260812 16:37:38.174266 14832 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d520c8ddc0ff4a1e9d90dfab76998e47: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:38.174578 14832 log.cc:824] T 00000000000000000000000000000000 P d520c8ddc0ff4a1e9d90dfab76998e47: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:38.175383 14832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d520c8ddc0ff4a1e9d90dfab76998e47: No bootstrap required, opened a new log
I20260812 16:37:38.177552 14832 raft_consensus.cc:348] T 00000000000000000000000000000000 P d520c8ddc0ff4a1e9d90dfab76998e47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d520c8ddc0ff4a1e9d90dfab76998e47" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 33909 } }
I20260812 16:37:38.177711 14832 raft_consensus.cc:374] T 00000000000000000000000000000000 P d520c8ddc0ff4a1e9d90dfab76998e47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:38.177763 14832 raft_consensus.cc:729] T 00000000000000000000000000000000 P d520c8ddc0ff4a1e9d90dfab76998e47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d520c8ddc0ff4a1e9d90dfab76998e47, State: Initialized, Role: FOLLOWER
I20260812 16:37:38.177917 14832 consensus_queue.cc:260] T 00000000000000000000000000000000 P d520c8ddc0ff4a1e9d90dfab76998e47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d520c8ddc0ff4a1e9d90dfab76998e47" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 33909 } }
I20260812 16:37:38.178000 14832 raft_consensus.cc:388] T 00000000000000000000000000000000 P d520c8ddc0ff4a1e9d90dfab76998e47 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:37:38.178054 14832 raft_consensus.cc:482] T 00000000000000000000000000000000 P d520c8ddc0ff4a1e9d90dfab76998e47 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:37:38.178107 14832 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d520c8ddc0ff4a1e9d90dfab76998e47 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:38.178121 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 14681
I20260812 16:37:38.178215 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:37:38.178437 1015 external_mini_cluster.cc:1428] Setting key 815fdb569164a54f7a57bd25acb889dd
I20260812 16:37:38.178742 14832 raft_consensus.cc:504] T 00000000000000000000000000000000 P d520c8ddc0ff4a1e9d90dfab76998e47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d520c8ddc0ff4a1e9d90dfab76998e47" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 33909 } }
I20260812 16:37:38.178874 14832 leader_election.cc:302] T 00000000000000000000000000000000 P d520c8ddc0ff4a1e9d90dfab76998e47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d520c8ddc0ff4a1e9d90dfab76998e47; no voters:
I20260812 16:37:38.179152 14832 leader_election.cc:288] T 00000000000000000000000000000000 P d520c8ddc0ff4a1e9d90dfab76998e47 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:37:38.179181 14839 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d520c8ddc0ff4a1e9d90dfab76998e47 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:37:38.179404 14839 raft_consensus.cc:686] T 00000000000000000000000000000000 P d520c8ddc0ff4a1e9d90dfab76998e47 [term 1 LEADER]: Becoming Leader. State: Replica: d520c8ddc0ff4a1e9d90dfab76998e47, State: Running, Role: LEADER
I20260812 16:37:38.179522 14832 sys_catalog.cc:564] T 00000000000000000000000000000000 P d520c8ddc0ff4a1e9d90dfab76998e47 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:37:38.179594 14839 consensus_queue.cc:237] T 00000000000000000000000000000000 P d520c8ddc0ff4a1e9d90dfab76998e47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d520c8ddc0ff4a1e9d90dfab76998e47" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 33909 } }
I20260812 16:37:38.180344 14841 sys_catalog.cc:455] T 00000000000000000000000000000000 P d520c8ddc0ff4a1e9d90dfab76998e47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d520c8ddc0ff4a1e9d90dfab76998e47. Latest consensus state: current_term: 1 leader_uuid: "d520c8ddc0ff4a1e9d90dfab76998e47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d520c8ddc0ff4a1e9d90dfab76998e47" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 33909 } } }
I20260812 16:37:38.180451 14841 sys_catalog.cc:458] T 00000000000000000000000000000000 P d520c8ddc0ff4a1e9d90dfab76998e47 [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:38.180346 14840 sys_catalog.cc:455] T 00000000000000000000000000000000 P d520c8ddc0ff4a1e9d90dfab76998e47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d520c8ddc0ff4a1e9d90dfab76998e47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d520c8ddc0ff4a1e9d90dfab76998e47" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 33909 } } }
I20260812 16:37:38.180646 14840 sys_catalog.cc:458] T 00000000000000000000000000000000 P d520c8ddc0ff4a1e9d90dfab76998e47 [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:38.181012 14855 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:37:38.181520 14855 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:37:38.183166 14855 catalog_manager.cc:1347] Generated new cluster ID: 1f2de3766e164d8dbe53f9f76a428643
I20260812 16:37:38.183224 14855 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 16:37:38 dist-test-slave-w4v5 krb5kdc[14663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM
I20260812 16:37:38.189977 14838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:38.178935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35230 (local address 127.0.253.254:33909)
0812 16:37:38.179205 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:38.179216 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:38.179258 (+ 42us) server_negotiation.cc:407] Connection header received
0812 16:37:38.179417 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:38.179426 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:38.179485 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:38.180190 (+ 705us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:38.180435 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.181903 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.182962 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.183520 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.183646 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:38.186039 (+ 2393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:38.186043 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:38.186044 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:38.186056 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:38.188365 (+ 2309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:38.188762 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:38.188766 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:38.188766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:38.188882 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:38.189098 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:38.189099 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:38.189099 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:38.189350 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:38.189549 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:38.189616 (+ 67us) server_negotiation.cc:299] Negotiation successful
0812 16:37:38.189760 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":86,"threads_started":1}
I20260812 16:37:38.202404 14855 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:37:38.203009 14855 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:37:38.211184 14855 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d520c8ddc0ff4a1e9d90dfab76998e47: Generated new TSK 0
I20260812 16:37:38.211424 14855 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.0.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.193.keytab.
Entry for principal oryx/127.0.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.193.keytab.
I20260812 16:37:38.247789 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.193:0
--local_ip_for_outbound_sockets=127.0.253.193
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:33909
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.193.keytab
--principal=oryx/127.0.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:41647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:38.334038 14875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:38.334260 14875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:38.334287 14875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:38.335556 14875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:38.335598 14875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:38.335619 14875 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 16:37:38.335637 14875 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.193
I20260812 16:37:38.337195 14875 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:41647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.193.keytab
--principal=oryx/127.0.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.253.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.253.254:33909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.193
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:38.337474 14875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:38.337724 14875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:38.342891 14889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:38.342896 14884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:38.342900 14893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:38.343293 14875 server_base.cc:1034] running on GCE node
I20260812 16:37:38.343478 14875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:38.343712 14875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:38.344879 14875 hybrid_clock.cc:648] HybridClock initialized: now 1786552658344868 us; error 36 us; skew 500 ppm
Aug 12 16:37:38 dist-test-slave-w4v5 krb5kdc[14663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552658, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:38.347795 14875 init.cc:377] Logged in from keytab as oryx/127.0.253.193@KRBTEST.COM (short username oryx)
I20260812 16:37:38.348351 14875 webserver.cc:460] Webserver started at http://127.0.253.193:42677/ using document root <none> and password file <none>
I20260812 16:37:38.348659 14875 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:38.348733 14875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:38.348891 14875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:38.350106 14875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-0/data/instance:
uuid: "a03e7efd0f5540a1a44160559e469cc1"
format_stamp: "Formatted at 2026-08-12 16:37:38 on dist-test-slave-w4v5"
server_key: "3354ffc2f11c5be6e250fa2a404d86bb"
server_key_iv: "e5ed4736722c856f2a13b23c72c63215"
server_key_version: "encryptionkey@0"
I20260812 16:37:38.350521 14875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance:
uuid: "a03e7efd0f5540a1a44160559e469cc1"
format_stamp: "Formatted at 2026-08-12 16:37:38 on dist-test-slave-w4v5"
server_key: "3354ffc2f11c5be6e250fa2a404d86bb"
server_key_iv: "e5ed4736722c856f2a13b23c72c63215"
server_key_version: "encryptionkey@0"
I20260812 16:37:38.352123 14875 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:37:38.352979 14906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:38.353245 14875 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:38.353377 14875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-0/wal
uuid: "a03e7efd0f5540a1a44160559e469cc1"
format_stamp: "Formatted at 2026-08-12 16:37:38 on dist-test-slave-w4v5"
server_key: "3354ffc2f11c5be6e250fa2a404d86bb"
server_key_iv: "e5ed4736722c856f2a13b23c72c63215"
server_key_version: "encryptionkey@0"
I20260812 16:37:38.353468 14875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:38.385586 14875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:38.388052 14875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:38.388176 14875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:38.388718 14875 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:38.388783 14875 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:38.388819 14875 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:38.388846 14875 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 16:37:38 dist-test-slave-w4v5 krb5kdc[14663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552658, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.253.193@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM
I20260812 16:37:38.396432 14875 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.193:45315
I20260812 16:37:38.396876 15105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.193:45315 every 8 connection(s)
I20260812 16:37:38.396951 14875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
I20260812 16:37:38.399049 14838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:38.389158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:54543 (local address 127.0.253.254:33909)
0812 16:37:38.389230 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:38.389232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:38.389502 (+ 270us) server_negotiation.cc:407] Connection header received
0812 16:37:38.390253 (+ 751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:38.390270 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:38.390288 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:38.390364 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:38.391326 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.391844 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.392891 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.393103 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.393197 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:38.395944 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:38.395945 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:38.395946 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:38.395957 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:38.397633 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:38.398064 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:38.398066 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:38.398066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:38.398130 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:38.398387 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:38.398388 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:38.398388 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:38.398509 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:38.398588 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:38.398862 (+ 274us) server_negotiation.cc:299] Negotiation successful
0812 16:37:38.398962 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 16:37:38.399250 14924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:38.389116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:33909 (local address 127.0.253.193:54543)
0812 16:37:38.389333 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:38.389389 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:38.390144 (+ 755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:38.390442 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:38.390447 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:38.390871 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:38.391215 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:38.391224 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.391955 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.391956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:38.392794 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:38.392797 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.393209 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.393210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:38.393346 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:38.393845 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:38.393856 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:38.395843 (+ 1987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:38.397755 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:38.397762 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:38.397764 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:38.397986 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:38.398208 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:38.398209 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:38.398209 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:38.398311 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:38.398579 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:38.398586 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:38.398718 (+ 132us) client_negotiation.cc:769] Sending connection context
0812 16:37:38.398826 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 16:37:38.398989 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":115,"threads_started":1}
I20260812 16:37:38.399838 15106 heartbeater.cc:347] Connected to a master server at 127.0.253.254:33909
I20260812 16:37:38.399922 15106 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:38.400120 15106 heartbeater.cc:511] Master 127.0.253.254:33909 requested a full tablet report, sending...
I20260812 16:37:38.400660 14753 ts_manager.cc:194] Registered new tserver with Master: a03e7efd0f5540a1a44160559e469cc1 (127.0.253.193:45315)
I20260812 16:37:38.401430 14753 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.253.193@KRBTEST.COM'} at 127.0.253.193:54543
I20260812 16:37:38.404172 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 14875
I20260812 16:37:38.404290 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance
I20260812 16:37:38.404534 1015 external_mini_cluster.cc:1428] Setting key 197ed5e8db3671ccc87ad0006a67ac91
I20260812 16:37:38.406695 15116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:38.400979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:33909 (local address 127.0.253.193:36555)
0812 16:37:38.401107 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:38.401119 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:38.401184 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:38.401307 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:38.401307 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:38.401481 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:38.401529 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:38.401549 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.402289 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.402290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:38.402802 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:38.402804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.403221 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.403222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:38.403272 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:38.403620 (+ 348us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:38.403624 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:38.403886 (+ 262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:38.405751 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:38.405752 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:38.405752 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:38.405924 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:38.406122 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:38.406123 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:38.406123 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:38.406178 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:38.406451 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:38.406452 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:38.406487 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 16:37:38.406535 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 16:37:38.406615 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":71,"threads_started":1}
I20260812 16:37:38.406800 14838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:38.401050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:36555 (local address 127.0.253.254:33909)
0812 16:37:38.401097 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:38.401099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:38.401168 (+ 69us) server_negotiation.cc:407] Connection header received
0812 16:37:38.401240 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:38.401241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:38.401251 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:38.401294 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:38.401613 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.402185 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.402895 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.403139 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.403223 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:38.403977 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:38.403978 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:38.403979 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:38.403988 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:38.405663 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:38.405992 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:38.405993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:38.405994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:38.406056 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:38.406241 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:38.406241 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:38.406242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:38.406378 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:38.406447 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:38.406583 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 16:37:38.406668 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
WARNING: no policy specified for oryx/127.0.253.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.253.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.194.keytab.
Entry for principal oryx/127.0.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.194.keytab.
I20260812 16:37:38.434319 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.194:0
--local_ip_for_outbound_sockets=127.0.253.194
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:33909
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.194.keytab
--principal=oryx/127.0.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:41647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:38.517273 15126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:38.517509 15126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:38.517547 15126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:38.518672 15126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:38.518736 15126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:38.518770 15126 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 16:37:38.518807 15126 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.194
I20260812 16:37:38.520365 15126 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:41647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.194.keytab
--principal=oryx/127.0.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.253.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.253.254:33909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15126
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.194
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:38.520653 15126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:38.520884 15126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:38.526010 15137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:38.526026 15134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:38.526065 15126 server_base.cc:1034] running on GCE node
W20260812 16:37:38.526026 15133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:38.526449 15126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:38.526707 15126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:38.527863 15126 hybrid_clock.cc:648] HybridClock initialized: now 1786552658527840 us; error 40 us; skew 500 ppm
Aug 12 16:37:38 dist-test-slave-w4v5 krb5kdc[14663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552658, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:38.538492 15126 init.cc:377] Logged in from keytab as oryx/127.0.253.194@KRBTEST.COM (short username oryx)
I20260812 16:37:38.539012 15126 webserver.cc:460] Webserver started at http://127.0.253.194:38737/ using document root <none> and password file <none>
I20260812 16:37:38.539335 15126 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:38.539397 15126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:38.539494 15126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:38.540493 15126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-1/data/instance:
uuid: "87509834f1fc44a69dc033288c16409d"
format_stamp: "Formatted at 2026-08-12 16:37:38 on dist-test-slave-w4v5"
server_key: "b95a99ea0c279f90df933d3c7f4242fd"
server_key_iv: "37f4df342839182eb9ba4cab85737139"
server_key_version: "encryptionkey@0"
I20260812 16:37:38.540805 15126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance:
uuid: "87509834f1fc44a69dc033288c16409d"
format_stamp: "Formatted at 2026-08-12 16:37:38 on dist-test-slave-w4v5"
server_key: "b95a99ea0c279f90df933d3c7f4242fd"
server_key_iv: "37f4df342839182eb9ba4cab85737139"
server_key_version: "encryptionkey@0"
I20260812 16:37:38.542119 15126 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:37:38.542873 15145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:38.543136 15126 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:38.543228 15126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-1/wal
uuid: "87509834f1fc44a69dc033288c16409d"
format_stamp: "Formatted at 2026-08-12 16:37:38 on dist-test-slave-w4v5"
server_key: "b95a99ea0c279f90df933d3c7f4242fd"
server_key_iv: "37f4df342839182eb9ba4cab85737139"
server_key_version: "encryptionkey@0"
I20260812 16:37:38.543285 15126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:38.561275 15126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:38.563725 15126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:38.563848 15126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:38.564339 15126 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:38.564379 15126 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:38.564402 15126 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:38.564414 15126 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:38.571270 15126 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.194:33615
I20260812 16:37:38.571359 15348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.194:33615 every 8 connection(s)
I20260812 16:37:38.571691 15126 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
Aug 12 16:37:38 dist-test-slave-w4v5 krb5kdc[14663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552658, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.253.194@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM
I20260812 16:37:38.575037 14838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:38.564873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:36295 (local address 127.0.253.254:33909)
0812 16:37:38.564950 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:38.564953 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:38.565175 (+ 222us) server_negotiation.cc:407] Connection header received
0812 16:37:38.565664 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:38.565666 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:38.565680 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:38.565750 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:38.566642 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.567511 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.569169 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.569499 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.569583 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:38.572366 (+ 2783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:38.572368 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:38.572368 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:38.572376 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:38.573801 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:38.574195 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:38.574196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:38.574196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:38.574246 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:38.574514 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:38.574515 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:38.574515 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:38.574614 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:38.574665 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:38.574913 (+ 248us) server_negotiation.cc:299] Negotiation successful
0812 16:37:38.574974 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 16:37:38.575304 15166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:38.564836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:33909 (local address 127.0.253.194:36295)
0812 16:37:38.565032 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:38.565073 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:38.565561 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:38.565818 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:38.565821 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:38.566205 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:38.566526 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:38.566533 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.567640 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.567642 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:38.569062 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:38.569065 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.569606 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.569608 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:38.569750 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:38.570385 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:38.570397 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:38.572245 (+ 1848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:38.573880 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:38.573887 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:38.573888 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:38.574121 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:38.574332 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:38.574333 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:38.574334 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:38.574448 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:38.574676 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:38.574681 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:38.574796 (+ 115us) client_negotiation.cc:769] Sending connection context
0812 16:37:38.574882 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 16:37:38.575005 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":104,"threads_started":1}
I20260812 16:37:38.575870 15349 heartbeater.cc:347] Connected to a master server at 127.0.253.254:33909
I20260812 16:37:38.575975 15349 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:38.576161 15349 heartbeater.cc:511] Master 127.0.253.254:33909 requested a full tablet report, sending...
I20260812 16:37:38.576603 14751 ts_manager.cc:194] Registered new tserver with Master: 87509834f1fc44a69dc033288c16409d (127.0.253.194:33615)
I20260812 16:37:38.577126 14751 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.253.194@KRBTEST.COM'} at 127.0.253.194:36295
I20260812 16:37:38.580452 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 15126
I20260812 16:37:38.580588 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance
I20260812 16:37:38.580884 1015 external_mini_cluster.cc:1428] Setting key 9370b3c0260db5baf5b91716556868d7
I20260812 16:37:38.583256 14838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:38.577102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:44109 (local address 127.0.253.254:33909)
0812 16:37:38.577326 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:38.577328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:38.577334 (+ 6us) server_negotiation.cc:407] Connection header received
0812 16:37:38.577357 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:38.577358 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:38.577366 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:38.577421 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:38.577772 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.578279 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.578970 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.579246 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.579320 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:38.580241 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:38.580242 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:38.580243 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:38.580249 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:38.581961 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:38.582332 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:38.582333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:38.582333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:38.582397 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:38.582585 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:38.582586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:38.582586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:38.582795 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:38.582867 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:38.583002 (+ 135us) server_negotiation.cc:299] Negotiation successful
0812 16:37:38.583123 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206}
I20260812 16:37:38.583338 15362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:38.577008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:33909 (local address 127.0.253.194:44109)
0812 16:37:38.577172 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:38.577183 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:38.577236 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:38.577441 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:38.577442 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:38.577618 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:38.577684 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:38.577686 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.578377 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.578378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:38.578887 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:38.578889 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.579320 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.579320 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:38.579386 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:38.579765 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:38.579768 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:38.580134 (+ 366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:38.582075 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:38.582076 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:38.582076 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:38.582255 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:38.582466 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:38.582467 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:38.582467 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:38.582520 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:38.582862 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:38.582862 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:38.582902 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 16:37:38.582954 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 16:37:38.583064 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":107,"threads_started":1}
WARNING: no policy specified for oryx/127.0.253.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.195.keytab.
Entry for principal oryx/127.0.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.195.keytab.
I20260812 16:37:38.609701 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.195:0
--local_ip_for_outbound_sockets=127.0.253.195
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:33909
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.195.keytab
--principal=oryx/127.0.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:41647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:38.693630 15370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:38.693841 15370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:38.693866 15370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:38.695024 15370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:38.695066 15370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:38.695180 15370 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 16:37:38.695219 15370 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.195
I20260812 16:37:38.696750 15370 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:41647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.195.keytab
--principal=oryx/127.0.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.253.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.253.254:33909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15370
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.195
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:38.696995 15370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:38.697254 15370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:38.702531 15378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:38.702514 15381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:38.702756 15370 server_base.cc:1034] running on GCE node
W20260812 16:37:38.702514 15379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:38.703131 15370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:38.703401 15370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:38.704622 15370 hybrid_clock.cc:648] HybridClock initialized: now 1786552658704601 us; error 26 us; skew 500 ppm
Aug 12 16:37:38 dist-test-slave-w4v5 krb5kdc[14663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552658, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:38.707365 15370 init.cc:377] Logged in from keytab as oryx/127.0.253.195@KRBTEST.COM (short username oryx)
I20260812 16:37:38.707804 15370 webserver.cc:460] Webserver started at http://127.0.253.195:40387/ using document root <none> and password file <none>
I20260812 16:37:38.707999 15370 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:38.708041 15370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:38.708132 15370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:38.709126 15370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-2/data/instance:
uuid: "fc4578a90e094dfd8f119ccd8ee2227a"
format_stamp: "Formatted at 2026-08-12 16:37:38 on dist-test-slave-w4v5"
server_key: "b0cf1569c5b4a18846f8623d1969bd15"
server_key_iv: "03404e84e13b47cb2b7b5c3d6f487c25"
server_key_version: "encryptionkey@0"
I20260812 16:37:38.709431 15370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance:
uuid: "fc4578a90e094dfd8f119ccd8ee2227a"
format_stamp: "Formatted at 2026-08-12 16:37:38 on dist-test-slave-w4v5"
server_key: "b0cf1569c5b4a18846f8623d1969bd15"
server_key_iv: "03404e84e13b47cb2b7b5c3d6f487c25"
server_key_version: "encryptionkey@0"
I20260812 16:37:38.710922 15370 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:37:38.711922 15393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:38.712095 15370 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:37:38.712186 15370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-2/wal
uuid: "fc4578a90e094dfd8f119ccd8ee2227a"
format_stamp: "Formatted at 2026-08-12 16:37:38 on dist-test-slave-w4v5"
server_key: "b0cf1569c5b4a18846f8623d1969bd15"
server_key_iv: "03404e84e13b47cb2b7b5c3d6f487c25"
server_key_version: "encryptionkey@0"
I20260812 16:37:38.712240 15370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:38.730624 15370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:38.733006 15370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:38.733126 15370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:38.733619 15370 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:38.733659 15370 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:38.733687 15370 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:38.733745 15370 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:38.740566 15370 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.195:38573
I20260812 16:37:38.740653 15608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.195:38573 every 8 connection(s)
Aug 12 16:37:38 dist-test-slave-w4v5 krb5kdc[14663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552658, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.253.195@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM
I20260812 16:37:38.740973 15370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
I20260812 16:37:38.744093 14838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:38.734209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:55871 (local address 127.0.253.254:33909)
0812 16:37:38.734295 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:38.734298 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:38.734452 (+ 154us) server_negotiation.cc:407] Connection header received
0812 16:37:38.734947 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:38.734949 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:38.734963 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:38.735053 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:38.736059 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.736876 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.738461 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.738779 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.738862 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:38.741608 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:38.741609 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:38.741610 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:38.741616 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:38.742840 (+ 1224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:38.743274 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:38.743275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:38.743275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:38.743325 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:38.743538 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:38.743539 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:38.743539 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:38.743631 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:38.743677 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:38.743945 (+ 268us) server_negotiation.cc:299] Negotiation successful
0812 16:37:38.744012 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 16:37:38.744233 15408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:38.734161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:33909 (local address 127.0.253.195:55871)
0812 16:37:38.734316 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:38.734357 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:38.734844 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:38.735202 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:38.735206 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:38.735632 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:38.735942 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:38.735949 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.736988 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.736988 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:38.738355 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:38.738360 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.738875 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.738876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:38.738980 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:38.739653 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:38.739665 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:38.741484 (+ 1819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:38.742916 (+ 1432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:38.742922 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:38.742924 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:38.743214 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:38.743383 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:38.743383 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:38.743384 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:38.743495 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:38.743695 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:38.743701 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:38.743836 (+ 135us) client_negotiation.cc:769] Sending connection context
0812 16:37:38.743916 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 16:37:38.744037 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":84,"threads_started":1}
I20260812 16:37:38.744756 15609 heartbeater.cc:347] Connected to a master server at 127.0.253.254:33909
I20260812 16:37:38.744845 15609 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:38.745065 15609 heartbeater.cc:511] Master 127.0.253.254:33909 requested a full tablet report, sending...
I20260812 16:37:38.745383 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 15370
I20260812 16:37:38.745491 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance
I20260812 16:37:38.745509 14751 ts_manager.cc:194] Registered new tserver with Master: fc4578a90e094dfd8f119ccd8ee2227a (127.0.253.195:38573)
I20260812 16:37:38.745682 1015 external_mini_cluster.cc:1428] Setting key 9ae53f43ef9e8ba26cd248173343973f
I20260812 16:37:38.746031 14751 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.253.195@KRBTEST.COM'} at 127.0.253.195:55871
I20260812 16:37:38.746663 1015 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:37:38.751924 14838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:38.745947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:49245 (local address 127.0.253.254:33909)
0812 16:37:38.745983 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:38.745984 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:38.746148 (+ 164us) server_negotiation.cc:407] Connection header received
0812 16:37:38.746246 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:38.746247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:38.746258 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:38.746288 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:38.746699 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.747349 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.748075 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.748293 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.748377 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:38.749312 (+ 935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:38.749314 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:38.749314 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:38.749321 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:38.750755 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:38.751137 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:38.751139 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:38.751139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:38.751195 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:38.751413 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:38.751414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:38.751414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:38.751523 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:38.751605 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:38.751758 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 16:37:38.751835 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 16:37:38.751932 15620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:38.745926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:33909 (local address 127.0.253.195:49245)
0812 16:37:38.746081 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:38.746092 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:38.746174 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:38.746347 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:38.746347 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:38.746553 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:38.746629 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:38.746631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.747441 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.747442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:38.747948 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:38.747949 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.748381 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.748382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:38.748448 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:38.748930 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:38.748935 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:38.749223 (+ 288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:38.750859 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:38.750860 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:38.750861 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:38.751039 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:38.751274 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:38.751275 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:38.751275 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:38.751335 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:38.751597 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:38.751598 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:38.751646 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 16:37:38.751733 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 16:37:38.751837 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":82,"threads_started":1}
Aug 12 16:37:38 dist-test-slave-w4v5 krb5kdc[14663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552658, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 16:37:38 dist-test-slave-w4v5 krb5kdc[14663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:38 dist-test-slave-w4v5 krb5kdc[14663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
W20260812 16:37:38.773855 14838 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:38.768341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35242 (local address 127.0.253.254:33909)
0812 16:37:38.768404 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:38.768406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:38.768417 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:37:38.768465 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:38.768466 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:38.768477 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:38.768552 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:38.769163 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.769656 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.770401 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.770612 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.770685 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:38.773766 (+ 3081us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35242: BlockingRecv error: recv got EOF from 127.0.0.1:35242 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 12 16:37:38 dist-test-slave-w4v5 krb5kdc[14663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552658, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM
I20260812 16:37:38.781563 14838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:38.774632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35248 (local address 127.0.253.254:33909)
0812 16:37:38.774697 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:38.774698 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:38.774707 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:37:38.774765 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:38.774766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:38.774773 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:38.774823 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:38.775405 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.775899 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.776751 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.776928 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.777016 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:38.779175 (+ 2159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:38.779177 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:38.779177 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:38.779183 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:38.780553 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:38.780856 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:38.780858 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:38.780858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:38.780910 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:38.781057 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:38.781057 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:38.781057 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:38.781168 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:38.781244 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:38.781389 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 16:37:38.781474 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 16:37:38.783202 14751 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35248:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:37:38.784716 14751 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:37:38.791383 15642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:38.787046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.195:38573 (local address 127.0.0.1:48312)
0812 16:37:38.787229 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:38.787275 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:38.787347 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:38.787956 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:38.787957 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:38.787959 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:38.788064 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:38.788067 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.788958 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.788959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:38.790102 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:38.790105 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.791025 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.791026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:38.791141 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:38.791159 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 16:37:38.791237 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 16:37:38.791292 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":14,"thread_start_us":120,"threads_started":1}
I20260812 16:37:38.791388 15650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:38.787353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48312 (local address 127.0.253.195:38573)
0812 16:37:38.787768 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:38.787771 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:38.787781 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:37:38.787841 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:38.787843 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:38.787857 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:38.788024 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:38.788130 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.788869 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.790190 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.790936 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.791024 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:38.791138 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:38.791254 (+ 116us) server_negotiation.cc:299] Negotiation successful
0812 16:37:38.791295 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":97,"threads_started":1}
I20260812 16:37:38.791718 15649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:38.787324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:45315 (local address 127.0.0.1:34898)
0812 16:37:38.787559 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:38.787575 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:38.787620 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:38.787963 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:38.787964 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:38.787965 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:38.788169 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:38.788171 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.789439 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.789440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:38.790416 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:38.790418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.791462 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.791463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:38.791542 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:38.791547 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:37:38.791595 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 16:37:38.791638 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":2,"thread_start_us":84,"threads_started":1}
I20260812 16:37:38.791781 15645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:38.787088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:33615 (local address 127.0.0.1:40278)
0812 16:37:38.787494 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:38.787505 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:38.787554 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:38.787941 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:38.787944 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:38.787947 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:38.788021 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:38.788023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.789484 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.789485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:38.790627 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:38.790630 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.791563 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.791564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:38.791620 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:38.791623 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:37:38.791651 (+ 28us) client_negotiation.cc:240] Negotiation successful
0812 16:37:38.791681 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":123,"threads_started":1}
I20260812 16:37:38.792090 15646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:38.787293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40278 (local address 127.0.253.194:33615)
0812 16:37:38.787724 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:38.787729 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:38.787737 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:37:38.787771 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:38.787773 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:38.787786 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:38.788131 (+ 345us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:38.788225 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.789367 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.790717 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.791457 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.791840 (+ 383us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:38.791935 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:38.791963 (+ 28us) server_negotiation.cc:299] Negotiation successful
0812 16:37:38.792002 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":103,"threads_started":1}
I20260812 16:37:38.792416 15480 tablet_service.cc:1467] Processing CreateTablet for tablet a207e2e755e646efb69b822353128fe8 (DEFAULT_TABLE table=test-table [id=fcaa6c044c9e42c4b611f59e3d0b1c16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:38.792814 15480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a207e2e755e646efb69b822353128fe8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:38.792730 15236 tablet_service.cc:1467] Processing CreateTablet for tablet a207e2e755e646efb69b822353128fe8 (DEFAULT_TABLE table=test-table [id=fcaa6c044c9e42c4b611f59e3d0b1c16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:38.793085 15236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a207e2e755e646efb69b822353128fe8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:38.796001 15652 tablet_bootstrap.cc:492] T a207e2e755e646efb69b822353128fe8 P fc4578a90e094dfd8f119ccd8ee2227a: Bootstrap starting.
I20260812 16:37:38.796193 15651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:38.787580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34898 (local address 127.0.253.193:45315)
0812 16:37:38.787728 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:38.787733 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:38.787744 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:37:38.787788 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:38.787790 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:38.787807 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:38.787892 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:38.788255 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.789339 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.790517 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:38.791283 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:38.791354 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:38.791424 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:38.796029 (+ 4605us) server_negotiation.cc:299] Negotiation successful
0812 16:37:38.796081 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":78,"threads_started":1}
I20260812 16:37:38.796638 15656 tablet_bootstrap.cc:492] T a207e2e755e646efb69b822353128fe8 P 87509834f1fc44a69dc033288c16409d: Bootstrap starting.
I20260812 16:37:38.796729 15652 tablet_bootstrap.cc:654] T a207e2e755e646efb69b822353128fe8 P fc4578a90e094dfd8f119ccd8ee2227a: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:38.796814 14997 tablet_service.cc:1467] Processing CreateTablet for tablet a207e2e755e646efb69b822353128fe8 (DEFAULT_TABLE table=test-table [id=fcaa6c044c9e42c4b611f59e3d0b1c16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:38.797078 15652 log.cc:824] T a207e2e755e646efb69b822353128fe8 P fc4578a90e094dfd8f119ccd8ee2227a: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:38.797163 14997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a207e2e755e646efb69b822353128fe8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:38.797322 15656 tablet_bootstrap.cc:654] T a207e2e755e646efb69b822353128fe8 P 87509834f1fc44a69dc033288c16409d: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:38.797675 15656 log.cc:824] T a207e2e755e646efb69b822353128fe8 P 87509834f1fc44a69dc033288c16409d: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:38.798022 15652 tablet_bootstrap.cc:492] T a207e2e755e646efb69b822353128fe8 P fc4578a90e094dfd8f119ccd8ee2227a: No bootstrap required, opened a new log
I20260812 16:37:38.798166 15652 ts_tablet_manager.cc:1397] T a207e2e755e646efb69b822353128fe8 P fc4578a90e094dfd8f119ccd8ee2227a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:37:38.799314 15656 tablet_bootstrap.cc:492] T a207e2e755e646efb69b822353128fe8 P 87509834f1fc44a69dc033288c16409d: No bootstrap required, opened a new log
I20260812 16:37:38.799396 15656 ts_tablet_manager.cc:1397] T a207e2e755e646efb69b822353128fe8 P 87509834f1fc44a69dc033288c16409d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 16:37:38.800685 15652 raft_consensus.cc:348] T a207e2e755e646efb69b822353128fe8 P fc4578a90e094dfd8f119ccd8ee2227a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } } peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } }
I20260812 16:37:38.800972 15652 raft_consensus.cc:374] T a207e2e755e646efb69b822353128fe8 P fc4578a90e094dfd8f119ccd8ee2227a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:38.801079 15652 raft_consensus.cc:729] T a207e2e755e646efb69b822353128fe8 P fc4578a90e094dfd8f119ccd8ee2227a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc4578a90e094dfd8f119ccd8ee2227a, State: Initialized, Role: FOLLOWER
I20260812 16:37:38.801294 15652 consensus_queue.cc:260] T a207e2e755e646efb69b822353128fe8 P fc4578a90e094dfd8f119ccd8ee2227a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } } peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } }
I20260812 16:37:38.801697 15652 ts_tablet_manager.cc:1428] T a207e2e755e646efb69b822353128fe8 P fc4578a90e094dfd8f119ccd8ee2227a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:37:38.801753 15609 heartbeater.cc:503] Master 127.0.253.254:33909 was elected leader, sending a full tablet report...
I20260812 16:37:38.803413 15662 tablet_bootstrap.cc:492] T a207e2e755e646efb69b822353128fe8 P a03e7efd0f5540a1a44160559e469cc1: Bootstrap starting.
I20260812 16:37:38.804085 15662 tablet_bootstrap.cc:654] T a207e2e755e646efb69b822353128fe8 P a03e7efd0f5540a1a44160559e469cc1: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:38.804078 15656 raft_consensus.cc:348] T a207e2e755e646efb69b822353128fe8 P 87509834f1fc44a69dc033288c16409d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } } peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } }
I20260812 16:37:38.804253 15656 raft_consensus.cc:374] T a207e2e755e646efb69b822353128fe8 P 87509834f1fc44a69dc033288c16409d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:38.804302 15656 raft_consensus.cc:729] T a207e2e755e646efb69b822353128fe8 P 87509834f1fc44a69dc033288c16409d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87509834f1fc44a69dc033288c16409d, State: Initialized, Role: FOLLOWER
I20260812 16:37:38.804438 15662 log.cc:824] T a207e2e755e646efb69b822353128fe8 P a03e7efd0f5540a1a44160559e469cc1: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:38.804426 15656 consensus_queue.cc:260] T a207e2e755e646efb69b822353128fe8 P 87509834f1fc44a69dc033288c16409d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } } peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } }
I20260812 16:37:38.804718 15656 ts_tablet_manager.cc:1428] T a207e2e755e646efb69b822353128fe8 P 87509834f1fc44a69dc033288c16409d: Time spent starting tablet: real 0.005s user 0.001s sys 0.002s
I20260812 16:37:38.804774 15349 heartbeater.cc:503] Master 127.0.253.254:33909 was elected leader, sending a full tablet report...
I20260812 16:37:38.805239 15662 tablet_bootstrap.cc:492] T a207e2e755e646efb69b822353128fe8 P a03e7efd0f5540a1a44160559e469cc1: No bootstrap required, opened a new log
I20260812 16:37:38.805325 15662 ts_tablet_manager.cc:1397] T a207e2e755e646efb69b822353128fe8 P a03e7efd0f5540a1a44160559e469cc1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:37:38.806820 15662 raft_consensus.cc:348] T a207e2e755e646efb69b822353128fe8 P a03e7efd0f5540a1a44160559e469cc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } } peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } }
I20260812 16:37:38.806934 15662 raft_consensus.cc:374] T a207e2e755e646efb69b822353128fe8 P a03e7efd0f5540a1a44160559e469cc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:38.806954 15662 raft_consensus.cc:729] T a207e2e755e646efb69b822353128fe8 P a03e7efd0f5540a1a44160559e469cc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a03e7efd0f5540a1a44160559e469cc1, State: Initialized, Role: FOLLOWER
I20260812 16:37:38.807093 15662 consensus_queue.cc:260] T a207e2e755e646efb69b822353128fe8 P a03e7efd0f5540a1a44160559e469cc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } } peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } }
I20260812 16:37:38.807382 15662 ts_tablet_manager.cc:1428] T a207e2e755e646efb69b822353128fe8 P a03e7efd0f5540a1a44160559e469cc1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:37:38.807412 15106 heartbeater.cc:503] Master 127.0.253.254:33909 was elected leader, sending a full tablet report...
W20260812 16:37:38.822341 15355 tablet.cc:2369] T a207e2e755e646efb69b822353128fe8 P 87509834f1fc44a69dc033288c16409d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 16:37:38.897868 15107 tablet.cc:2369] T a207e2e755e646efb69b822353128fe8 P a03e7efd0f5540a1a44160559e469cc1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 16:37:38.991726 15610 tablet.cc:2369] T a207e2e755e646efb69b822353128fe8 P fc4578a90e094dfd8f119ccd8ee2227a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:37:39.027449 15660 raft_consensus.cc:482] T a207e2e755e646efb69b822353128fe8 P fc4578a90e094dfd8f119ccd8ee2227a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:37:39.027597 15660 raft_consensus.cc:504] T a207e2e755e646efb69b822353128fe8 P fc4578a90e094dfd8f119ccd8ee2227a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } } peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } }
I20260812 16:37:39.028028 15660 leader_election.cc:288] T a207e2e755e646efb69b822353128fe8 P fc4578a90e094dfd8f119ccd8ee2227a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87509834f1fc44a69dc033288c16409d (127.0.253.194:33615), a03e7efd0f5540a1a44160559e469cc1 (127.0.253.193:45315)
I20260812 16:37:39.031356 15646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:39.028417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:54333 (local address 127.0.253.194:33615)
0812 16:37:39.028472 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:39.028474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:39.028487 (+ 13us) server_negotiation.cc:407] Connection header received
0812 16:37:39.028624 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:39.028625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:39.028638 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:39.028702 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:39.028921 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.029461 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.030455 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.030949 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.031007 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:39.031015 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:39.031266 (+ 251us) server_negotiation.cc:299] Negotiation successful
0812 16:37:39.031300 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 16:37:39.031375 15408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:39.028305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:33615 (local address 127.0.253.195:54333)
0812 16:37:39.028394 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:39.028410 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:39.028485 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:39.028749 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:39.028750 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:39.028753 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:39.028836 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:39.028839 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.029545 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.029546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:39.030352 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:39.030355 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.031043 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.031044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:39.031183 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:39.031188 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:37:39.031251 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 16:37:39.031297 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260812 16:37:39.031832 15651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:39.028532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:37769 (local address 127.0.253.193:45315)
0812 16:37:39.028586 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:39.028588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:39.028737 (+ 149us) server_negotiation.cc:407] Connection header received
0812 16:37:39.028772 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:39.028773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:39.028784 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:39.028856 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:39.029068 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.029555 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.030804 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.031362 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.031433 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:39.031442 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:39.031745 (+ 303us) server_negotiation.cc:299] Negotiation successful
0812 16:37:39.031784 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 16:37:39.031781 15268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a207e2e755e646efb69b822353128fe8" candidate_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87509834f1fc44a69dc033288c16409d" is_pre_election: true
I20260812 16:37:39.031870 15673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:39.028451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:45315 (local address 127.0.253.195:37769)
0812 16:37:39.028619 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:39.028628 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:39.028722 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:39.028894 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:39.028895 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:39.028897 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:39.028976 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:39.028977 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.029806 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.029807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:39.030660 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:39.030662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.031557 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.031558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:39.031662 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:39.031667 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:37:39.031737 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 16:37:39.031775 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":103,"threads_started":1}
I20260812 16:37:39.031958 15268 raft_consensus.cc:2454] T a207e2e755e646efb69b822353128fe8 P 87509834f1fc44a69dc033288c16409d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc4578a90e094dfd8f119ccd8ee2227a in term 0.
I20260812 16:37:39.032212 15401 leader_election.cc:302] T a207e2e755e646efb69b822353128fe8 P fc4578a90e094dfd8f119ccd8ee2227a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87509834f1fc44a69dc033288c16409d, fc4578a90e094dfd8f119ccd8ee2227a; no voters:
I20260812 16:37:39.032282 15036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a207e2e755e646efb69b822353128fe8" candidate_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a03e7efd0f5540a1a44160559e469cc1" is_pre_election: true
I20260812 16:37:39.032430 15036 raft_consensus.cc:2454] T a207e2e755e646efb69b822353128fe8 P a03e7efd0f5540a1a44160559e469cc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc4578a90e094dfd8f119ccd8ee2227a in term 0.
I20260812 16:37:39.032408 15660 raft_consensus.cc:2781] T a207e2e755e646efb69b822353128fe8 P fc4578a90e094dfd8f119ccd8ee2227a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:37:39.032505 15660 raft_consensus.cc:482] T a207e2e755e646efb69b822353128fe8 P fc4578a90e094dfd8f119ccd8ee2227a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:37:39.032546 15660 raft_consensus.cc:3037] T a207e2e755e646efb69b822353128fe8 P fc4578a90e094dfd8f119ccd8ee2227a [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:39.033795 15660 raft_consensus.cc:504] T a207e2e755e646efb69b822353128fe8 P fc4578a90e094dfd8f119ccd8ee2227a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } } peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } }
I20260812 16:37:39.033948 15660 leader_election.cc:288] T a207e2e755e646efb69b822353128fe8 P fc4578a90e094dfd8f119ccd8ee2227a [CANDIDATE]: Term 1 election: Requested vote from peers 87509834f1fc44a69dc033288c16409d (127.0.253.194:33615), a03e7efd0f5540a1a44160559e469cc1 (127.0.253.193:45315)
I20260812 16:37:39.034159 15036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a207e2e755e646efb69b822353128fe8" candidate_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a03e7efd0f5540a1a44160559e469cc1"
I20260812 16:37:39.034214 15268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a207e2e755e646efb69b822353128fe8" candidate_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87509834f1fc44a69dc033288c16409d"
I20260812 16:37:39.034256 15036 raft_consensus.cc:3037] T a207e2e755e646efb69b822353128fe8 P a03e7efd0f5540a1a44160559e469cc1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:39.034339 15268 raft_consensus.cc:3037] T a207e2e755e646efb69b822353128fe8 P 87509834f1fc44a69dc033288c16409d [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:39.034991 15036 raft_consensus.cc:2454] T a207e2e755e646efb69b822353128fe8 P a03e7efd0f5540a1a44160559e469cc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc4578a90e094dfd8f119ccd8ee2227a in term 1.
I20260812 16:37:39.035193 15401 leader_election.cc:302] T a207e2e755e646efb69b822353128fe8 P fc4578a90e094dfd8f119ccd8ee2227a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a03e7efd0f5540a1a44160559e469cc1, fc4578a90e094dfd8f119ccd8ee2227a; no voters:
I20260812 16:37:39.035238 15268 raft_consensus.cc:2454] T a207e2e755e646efb69b822353128fe8 P 87509834f1fc44a69dc033288c16409d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc4578a90e094dfd8f119ccd8ee2227a in term 1.
I20260812 16:37:39.035285 15660 raft_consensus.cc:2781] T a207e2e755e646efb69b822353128fe8 P fc4578a90e094dfd8f119ccd8ee2227a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:37:39.035437 15660 raft_consensus.cc:686] T a207e2e755e646efb69b822353128fe8 P fc4578a90e094dfd8f119ccd8ee2227a [term 1 LEADER]: Becoming Leader. State: Replica: fc4578a90e094dfd8f119ccd8ee2227a, State: Running, Role: LEADER
I20260812 16:37:39.035545 15660 consensus_queue.cc:237] T a207e2e755e646efb69b822353128fe8 P fc4578a90e094dfd8f119ccd8ee2227a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } } peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } }
I20260812 16:37:39.036486 14751 catalog_manager.cc:5495] T a207e2e755e646efb69b822353128fe8 P fc4578a90e094dfd8f119ccd8ee2227a reported cstate change: term changed from 0 to 1, leader changed from <none> to fc4578a90e094dfd8f119ccd8ee2227a (127.0.253.195). New cstate: current_term: 1 leader_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } health_report { overall_health: UNKNOWN } } }
I20260812 16:37:39.079020 15650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:39.076487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48316 (local address 127.0.253.195:38573)
0812 16:37:39.076578 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:39.076580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:39.076588 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:37:39.076673 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:39.076674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:39.076694 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:39.076772 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:37:39.076938 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.077422 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.078202 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.078395 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.078470 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:39.078665 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:37:39.078771 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:37:39.078860 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:39.078903 (+ 43us) server_negotiation.cc:299] Negotiation successful
0812 16:37:39.078925 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 16:37:39.081213 15036 raft_consensus.cc:1261] T a207e2e755e646efb69b822353128fe8 P a03e7efd0f5540a1a44160559e469cc1 [term 1 FOLLOWER]: Refusing update from remote peer fc4578a90e094dfd8f119ccd8ee2227a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:37:39.081305 15268 raft_consensus.cc:1261] T a207e2e755e646efb69b822353128fe8 P 87509834f1fc44a69dc033288c16409d [term 1 FOLLOWER]: Refusing update from remote peer fc4578a90e094dfd8f119ccd8ee2227a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:37:39.081673 15660 consensus_queue.cc:1035] T a207e2e755e646efb69b822353128fe8 P fc4578a90e094dfd8f119ccd8ee2227a [LEADER]: Connected to new peer: Peer: permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:37:39.081869 15682 consensus_queue.cc:1035] T a207e2e755e646efb69b822353128fe8 P fc4578a90e094dfd8f119ccd8ee2227a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:37:39.084370 15687 mvcc.cc:205] Tried to move back new op lower bound from 7317719691593367552 to 7317719691410210816. Current Snapshot: MvccSnapshot[applied={T|T < 7317719691593367552}]
I20260812 16:37:39.086498 14753 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:37:39.086611 14753 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:37:39.087414 14753 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d520c8ddc0ff4a1e9d90dfab76998e47: Sending DeleteTablet for 3 replicas of tablet a207e2e755e646efb69b822353128fe8
I20260812 16:37:39.087728 15236 tablet_service.cc:1514] Processing DeleteTablet for tablet a207e2e755e646efb69b822353128fe8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:39 UTC) from {username='oryx', principal='oryx/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:40278
I20260812 16:37:39.087929 14997 tablet_service.cc:1514] Processing DeleteTablet for tablet a207e2e755e646efb69b822353128fe8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:39 UTC) from {username='oryx', principal='oryx/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:34898
I20260812 16:37:39.087946 14753 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35248:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:37:39.088084 14753 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:37:39.088119 15480 tablet_service.cc:1514] Processing DeleteTablet for tablet a207e2e755e646efb69b822353128fe8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:39 UTC) from {username='oryx', principal='oryx/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:48312
I20260812 16:37:39.088145 15712 tablet_replica.cc:330] T a207e2e755e646efb69b822353128fe8 P a03e7efd0f5540a1a44160559e469cc1: stopping tablet replica
I20260812 16:37:39.088241 15712 raft_consensus.cc:2229] T a207e2e755e646efb69b822353128fe8 P a03e7efd0f5540a1a44160559e469cc1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:37:39.088325 15713 tablet_replica.cc:330] T a207e2e755e646efb69b822353128fe8 P fc4578a90e094dfd8f119ccd8ee2227a: stopping tablet replica
I20260812 16:37:39.088342 15712 raft_consensus.cc:2258] T a207e2e755e646efb69b822353128fe8 P a03e7efd0f5540a1a44160559e469cc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:39.088407 15713 raft_consensus.cc:2229] T a207e2e755e646efb69b822353128fe8 P fc4578a90e094dfd8f119ccd8ee2227a [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:37:39.088506 15713 raft_consensus.cc:2258] T a207e2e755e646efb69b822353128fe8 P fc4578a90e094dfd8f119ccd8ee2227a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:39.088860 15712 ts_tablet_manager.cc:1905] T a207e2e755e646efb69b822353128fe8 P a03e7efd0f5540a1a44160559e469cc1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:37:39.088889 15713 ts_tablet_manager.cc:1905] T a207e2e755e646efb69b822353128fe8 P fc4578a90e094dfd8f119ccd8ee2227a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:37:39.089758 15711 tablet_replica.cc:330] T a207e2e755e646efb69b822353128fe8 P 87509834f1fc44a69dc033288c16409d: stopping tablet replica
I20260812 16:37:39.089915 15711 raft_consensus.cc:2229] T a207e2e755e646efb69b822353128fe8 P 87509834f1fc44a69dc033288c16409d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:37:39.090031 14997 tablet_service.cc:1467] Processing CreateTablet for tablet 825fc47862f54395bc7702747d526259 (DEFAULT_TABLE table=test-table [id=2715a40656434621aee6a29331e7e2e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:39.090077 15711 raft_consensus.cc:2258] T a207e2e755e646efb69b822353128fe8 P 87509834f1fc44a69dc033288c16409d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:39.090155 14997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 825fc47862f54395bc7702747d526259. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:39.090618 15713 ts_tablet_manager.cc:1918] T a207e2e755e646efb69b822353128fe8 P fc4578a90e094dfd8f119ccd8ee2227a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:37:39.090701 15713 log.cc:1196] T a207e2e755e646efb69b822353128fe8 P fc4578a90e094dfd8f119ccd8ee2227a: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-2/wal/wals/a207e2e755e646efb69b822353128fe8
I20260812 16:37:39.090758 15236 tablet_service.cc:1467] Processing CreateTablet for tablet 825fc47862f54395bc7702747d526259 (DEFAULT_TABLE table=test-table [id=2715a40656434621aee6a29331e7e2e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:39.090862 15712 ts_tablet_manager.cc:1918] T a207e2e755e646efb69b822353128fe8 P a03e7efd0f5540a1a44160559e469cc1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:37:39.090893 15236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 825fc47862f54395bc7702747d526259. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:39.090982 15712 log.cc:1196] T a207e2e755e646efb69b822353128fe8 P a03e7efd0f5540a1a44160559e469cc1: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-0/wal/wals/a207e2e755e646efb69b822353128fe8
I20260812 16:37:39.090996 15713 ts_tablet_manager.cc:1939] T a207e2e755e646efb69b822353128fe8 P fc4578a90e094dfd8f119ccd8ee2227a: Deleting consensus metadata
I20260812 16:37:39.091311 15712 ts_tablet_manager.cc:1939] T a207e2e755e646efb69b822353128fe8 P a03e7efd0f5540a1a44160559e469cc1: Deleting consensus metadata
I20260812 16:37:39.091462 14718 catalog_manager.cc:4841] TS fc4578a90e094dfd8f119ccd8ee2227a (127.0.253.195:38573): tablet a207e2e755e646efb69b822353128fe8 (table test-table [id=fcaa6c044c9e42c4b611f59e3d0b1c16]) successfully deleted
I20260812 16:37:39.091792 14721 catalog_manager.cc:4841] TS a03e7efd0f5540a1a44160559e469cc1 (127.0.253.193:45315): tablet a207e2e755e646efb69b822353128fe8 (table test-table [id=fcaa6c044c9e42c4b611f59e3d0b1c16]) successfully deleted
I20260812 16:37:39.092280 15480 tablet_service.cc:1467] Processing CreateTablet for tablet 825fc47862f54395bc7702747d526259 (DEFAULT_TABLE table=test-table [id=2715a40656434621aee6a29331e7e2e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:39.092330 15656 tablet_bootstrap.cc:492] T 825fc47862f54395bc7702747d526259 P 87509834f1fc44a69dc033288c16409d: Bootstrap starting.
I20260812 16:37:39.092408 15480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 825fc47862f54395bc7702747d526259. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:39.092720 15656 tablet_bootstrap.cc:654] T 825fc47862f54395bc7702747d526259 P 87509834f1fc44a69dc033288c16409d: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:39.093520 15656 tablet_bootstrap.cc:492] T 825fc47862f54395bc7702747d526259 P 87509834f1fc44a69dc033288c16409d: No bootstrap required, opened a new log
I20260812 16:37:39.093520 15652 tablet_bootstrap.cc:492] T 825fc47862f54395bc7702747d526259 P fc4578a90e094dfd8f119ccd8ee2227a: Bootstrap starting.
I20260812 16:37:39.093573 15656 ts_tablet_manager.cc:1397] T 825fc47862f54395bc7702747d526259 P 87509834f1fc44a69dc033288c16409d: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260812 16:37:39.093762 15656 raft_consensus.cc:348] T 825fc47862f54395bc7702747d526259 P 87509834f1fc44a69dc033288c16409d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } }
I20260812 16:37:39.093832 15656 raft_consensus.cc:374] T 825fc47862f54395bc7702747d526259 P 87509834f1fc44a69dc033288c16409d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:39.093856 15656 raft_consensus.cc:729] T 825fc47862f54395bc7702747d526259 P 87509834f1fc44a69dc033288c16409d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87509834f1fc44a69dc033288c16409d, State: Initialized, Role: FOLLOWER
I20260812 16:37:39.093912 15656 consensus_queue.cc:260] T 825fc47862f54395bc7702747d526259 P 87509834f1fc44a69dc033288c16409d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } }
I20260812 16:37:39.093940 15652 tablet_bootstrap.cc:654] T 825fc47862f54395bc7702747d526259 P fc4578a90e094dfd8f119ccd8ee2227a: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:39.094050 15656 ts_tablet_manager.cc:1428] T 825fc47862f54395bc7702747d526259 P 87509834f1fc44a69dc033288c16409d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:39.094297 15662 tablet_bootstrap.cc:492] T 825fc47862f54395bc7702747d526259 P a03e7efd0f5540a1a44160559e469cc1: Bootstrap starting.
I20260812 16:37:39.094687 15662 tablet_bootstrap.cc:654] T 825fc47862f54395bc7702747d526259 P a03e7efd0f5540a1a44160559e469cc1: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:39.095160 15652 tablet_bootstrap.cc:492] T 825fc47862f54395bc7702747d526259 P fc4578a90e094dfd8f119ccd8ee2227a: No bootstrap required, opened a new log
I20260812 16:37:39.095238 15652 ts_tablet_manager.cc:1397] T 825fc47862f54395bc7702747d526259 P fc4578a90e094dfd8f119ccd8ee2227a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:37:39.095397 15652 raft_consensus.cc:348] T 825fc47862f54395bc7702747d526259 P fc4578a90e094dfd8f119ccd8ee2227a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } }
I20260812 16:37:39.095472 15652 raft_consensus.cc:374] T 825fc47862f54395bc7702747d526259 P fc4578a90e094dfd8f119ccd8ee2227a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:39.095494 15652 raft_consensus.cc:729] T 825fc47862f54395bc7702747d526259 P fc4578a90e094dfd8f119ccd8ee2227a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc4578a90e094dfd8f119ccd8ee2227a, State: Initialized, Role: FOLLOWER
I20260812 16:37:39.095491 15662 tablet_bootstrap.cc:492] T 825fc47862f54395bc7702747d526259 P a03e7efd0f5540a1a44160559e469cc1: No bootstrap required, opened a new log
I20260812 16:37:39.095543 15662 ts_tablet_manager.cc:1397] T 825fc47862f54395bc7702747d526259 P a03e7efd0f5540a1a44160559e469cc1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:39.095533 15652 consensus_queue.cc:260] T 825fc47862f54395bc7702747d526259 P fc4578a90e094dfd8f119ccd8ee2227a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } }
I20260812 16:37:39.095613 15652 ts_tablet_manager.cc:1428] T 825fc47862f54395bc7702747d526259 P fc4578a90e094dfd8f119ccd8ee2227a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 16:37:39.095677 15662 raft_consensus.cc:348] T 825fc47862f54395bc7702747d526259 P a03e7efd0f5540a1a44160559e469cc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } }
I20260812 16:37:39.095746 15662 raft_consensus.cc:374] T 825fc47862f54395bc7702747d526259 P a03e7efd0f5540a1a44160559e469cc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:39.095772 15662 raft_consensus.cc:729] T 825fc47862f54395bc7702747d526259 P a03e7efd0f5540a1a44160559e469cc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a03e7efd0f5540a1a44160559e469cc1, State: Initialized, Role: FOLLOWER
I20260812 16:37:39.095816 15662 consensus_queue.cc:260] T 825fc47862f54395bc7702747d526259 P a03e7efd0f5540a1a44160559e469cc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } }
I20260812 16:37:39.095927 15662 ts_tablet_manager.cc:1428] T 825fc47862f54395bc7702747d526259 P a03e7efd0f5540a1a44160559e469cc1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 16:37:39.102603 15711 ts_tablet_manager.cc:1905] T a207e2e755e646efb69b822353128fe8 P 87509834f1fc44a69dc033288c16409d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:37:39.104393 15711 ts_tablet_manager.cc:1918] T a207e2e755e646efb69b822353128fe8 P 87509834f1fc44a69dc033288c16409d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:37:39.104491 15711 log.cc:1196] T a207e2e755e646efb69b822353128fe8 P 87509834f1fc44a69dc033288c16409d: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-1/wal/wals/a207e2e755e646efb69b822353128fe8
I20260812 16:37:39.105108 15711 ts_tablet_manager.cc:1939] T a207e2e755e646efb69b822353128fe8 P 87509834f1fc44a69dc033288c16409d: Deleting consensus metadata
I20260812 16:37:39.105573 14721 catalog_manager.cc:4841] TS 87509834f1fc44a69dc033288c16409d (127.0.253.194:33615): tablet a207e2e755e646efb69b822353128fe8 (table test-table [id=fcaa6c044c9e42c4b611f59e3d0b1c16]) successfully deleted
I20260812 16:37:39.286696 15663 raft_consensus.cc:482] T 825fc47862f54395bc7702747d526259 P 87509834f1fc44a69dc033288c16409d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:37:39.286813 15663 raft_consensus.cc:504] T 825fc47862f54395bc7702747d526259 P 87509834f1fc44a69dc033288c16409d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } }
I20260812 16:37:39.287289 15663 leader_election.cc:288] T 825fc47862f54395bc7702747d526259 P 87509834f1fc44a69dc033288c16409d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a03e7efd0f5540a1a44160559e469cc1 (127.0.253.193:45315), fc4578a90e094dfd8f119ccd8ee2227a (127.0.253.195:38573)
I20260812 16:37:39.290385 15651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:39.287720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:39295 (local address 127.0.253.193:45315)
0812 16:37:39.287785 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:39.287787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:39.287800 (+ 13us) server_negotiation.cc:407] Connection header received
0812 16:37:39.287837 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:39.287837 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:39.287844 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:39.287887 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:39.288140 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.288612 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.289650 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.290008 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.290068 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:39.290075 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:39.290260 (+ 185us) server_negotiation.cc:299] Negotiation successful
0812 16:37:39.290308 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 16:37:39.290416 15650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:39.287738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:50157 (local address 127.0.253.195:38573)
0812 16:37:39.287792 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:39.287794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:39.287804 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:37:39.287834 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:39.287835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:39.287845 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:39.287929 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:39.288111 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.288606 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.289652 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.290021 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.290115 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:39.290124 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:39.290309 (+ 185us) server_negotiation.cc:299] Negotiation successful
0812 16:37:39.290350 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 16:37:39.290421 15717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:39.287521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:45315 (local address 127.0.253.194:39295)
0812 16:37:39.287667 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:39.287680 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:39.287757 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:39.287914 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:39.287915 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:39.287917 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:39.288038 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:39.288040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.288714 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.288715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:39.289551 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:39.289553 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.290093 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.290094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:39.290184 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:39.290188 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:37:39.290233 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 16:37:39.290264 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":28,"thread_start_us":70,"threads_started":1}
I20260812 16:37:39.290436 15718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:39.287517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.195:38573 (local address 127.0.253.194:50157)
0812 16:37:39.287659 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:39.287671 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:39.287735 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:39.287956 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:39.287958 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:39.287960 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:39.288030 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:39.288032 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.288689 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.288691 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:39.289546 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:39.289549 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.290160 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.290161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:39.290211 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:39.290214 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:37:39.290291 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 16:37:39.290327 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":46,"thread_start_us":70,"threads_started":1}
I20260812 16:37:39.290865 15036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "825fc47862f54395bc7702747d526259" candidate_uuid: "87509834f1fc44a69dc033288c16409d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a03e7efd0f5540a1a44160559e469cc1" is_pre_election: true
I20260812 16:37:39.290969 15036 raft_consensus.cc:2454] T 825fc47862f54395bc7702747d526259 P a03e7efd0f5540a1a44160559e469cc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87509834f1fc44a69dc033288c16409d in term 0.
I20260812 16:37:39.290956 15516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "825fc47862f54395bc7702747d526259" candidate_uuid: "87509834f1fc44a69dc033288c16409d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" is_pre_election: true
I20260812 16:37:39.291144 15516 raft_consensus.cc:2454] T 825fc47862f54395bc7702747d526259 P fc4578a90e094dfd8f119ccd8ee2227a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87509834f1fc44a69dc033288c16409d in term 0.
I20260812 16:37:39.291172 15153 leader_election.cc:302] T 825fc47862f54395bc7702747d526259 P 87509834f1fc44a69dc033288c16409d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87509834f1fc44a69dc033288c16409d, a03e7efd0f5540a1a44160559e469cc1; no voters:
I20260812 16:37:39.291306 15663 raft_consensus.cc:2781] T 825fc47862f54395bc7702747d526259 P 87509834f1fc44a69dc033288c16409d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:37:39.291345 15663 raft_consensus.cc:482] T 825fc47862f54395bc7702747d526259 P 87509834f1fc44a69dc033288c16409d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:37:39.291369 15663 raft_consensus.cc:3037] T 825fc47862f54395bc7702747d526259 P 87509834f1fc44a69dc033288c16409d [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:39.292074 15663 raft_consensus.cc:504] T 825fc47862f54395bc7702747d526259 P 87509834f1fc44a69dc033288c16409d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } }
I20260812 16:37:39.292230 15663 leader_election.cc:288] T 825fc47862f54395bc7702747d526259 P 87509834f1fc44a69dc033288c16409d [CANDIDATE]: Term 1 election: Requested vote from peers a03e7efd0f5540a1a44160559e469cc1 (127.0.253.193:45315), fc4578a90e094dfd8f119ccd8ee2227a (127.0.253.195:38573)
I20260812 16:37:39.292438 15516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "825fc47862f54395bc7702747d526259" candidate_uuid: "87509834f1fc44a69dc033288c16409d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc4578a90e094dfd8f119ccd8ee2227a"
I20260812 16:37:39.292532 15516 raft_consensus.cc:3037] T 825fc47862f54395bc7702747d526259 P fc4578a90e094dfd8f119ccd8ee2227a [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:39.292726 15036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "825fc47862f54395bc7702747d526259" candidate_uuid: "87509834f1fc44a69dc033288c16409d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a03e7efd0f5540a1a44160559e469cc1"
I20260812 16:37:39.292824 15036 raft_consensus.cc:3037] T 825fc47862f54395bc7702747d526259 P a03e7efd0f5540a1a44160559e469cc1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:39.293304 15516 raft_consensus.cc:2454] T 825fc47862f54395bc7702747d526259 P fc4578a90e094dfd8f119ccd8ee2227a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87509834f1fc44a69dc033288c16409d in term 1.
I20260812 16:37:39.293444 15036 raft_consensus.cc:2454] T 825fc47862f54395bc7702747d526259 P a03e7efd0f5540a1a44160559e469cc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87509834f1fc44a69dc033288c16409d in term 1.
I20260812 16:37:39.293551 15152 leader_election.cc:302] T 825fc47862f54395bc7702747d526259 P 87509834f1fc44a69dc033288c16409d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87509834f1fc44a69dc033288c16409d, fc4578a90e094dfd8f119ccd8ee2227a; no voters:
I20260812 16:37:39.293659 15663 raft_consensus.cc:2781] T 825fc47862f54395bc7702747d526259 P 87509834f1fc44a69dc033288c16409d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:37:39.293851 15663 raft_consensus.cc:686] T 825fc47862f54395bc7702747d526259 P 87509834f1fc44a69dc033288c16409d [term 1 LEADER]: Becoming Leader. State: Replica: 87509834f1fc44a69dc033288c16409d, State: Running, Role: LEADER
I20260812 16:37:39.293943 15663 consensus_queue.cc:237] T 825fc47862f54395bc7702747d526259 P 87509834f1fc44a69dc033288c16409d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } }
I20260812 16:37:39.294678 14753 catalog_manager.cc:5495] T 825fc47862f54395bc7702747d526259 P 87509834f1fc44a69dc033288c16409d reported cstate change: term changed from 0 to 1, leader changed from <none> to 87509834f1fc44a69dc033288c16409d (127.0.253.194). New cstate: current_term: 1 leader_uuid: "87509834f1fc44a69dc033288c16409d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } health_report { overall_health: UNKNOWN } } }
Aug 12 16:37:39 dist-test-slave-w4v5 krb5kdc[14663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552658, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.253.254@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM
I20260812 16:37:39.352051 15741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:39.343968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35252 (local address 127.0.253.254:33909)
0812 16:37:39.344131 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:39.344133 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:39.344209 (+ 76us) server_negotiation.cc:407] Connection header received
0812 16:37:39.344304 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:39.344305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:39.344316 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:39.344404 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:39.344989 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.345556 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.346325 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.346540 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.346621 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:39.348952 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:39.348953 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:39.348953 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:39.348960 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:39.350633 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:39.351211 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:39.351212 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:39.351213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:39.351277 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:39.351478 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:39.351479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:39.351479 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:39.351596 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:39.351671 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:39.351835 (+ 164us) server_negotiation.cc:299] Negotiation successful
0812 16:37:39.351929 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":73,"threads_started":1}
I20260812 16:37:39.352116 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:39.343875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:33909 (local address 127.0.0.1:35252)
0812 16:37:39.344117 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:39.344130 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:39.344231 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:39.344419 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:39.344420 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:39.344843 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:39.344900 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:39.344903 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.345637 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.345639 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:39.346238 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:39.346243 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.346620 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.346620 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:39.346672 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:39.347238 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:39.347243 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:39.348867 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:39.350767 (+ 1900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:39.350775 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:39.350777 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:39.351126 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:39.351342 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:39.351343 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:39.351344 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:39.351406 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:39.351661 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:39.351670 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:39.351723 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 16:37:39.351809 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 16:37:39.351996 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":84,"thread_start_us":139,"threads_started":1}
I20260812 16:37:39.354092 14753 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:35252:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 16:37:39.354341 14753 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:37:39.355980 15480 tablet_service.cc:1467] Processing CreateTablet for tablet 3a837cdca46942cda0e9548106323de7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2b6fdb197f08494da131b410570352df]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 16:37:39.356005 15236 tablet_service.cc:1467] Processing CreateTablet for tablet 3a837cdca46942cda0e9548106323de7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2b6fdb197f08494da131b410570352df]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 16:37:39.355991 14997 tablet_service.cc:1467] Processing CreateTablet for tablet 3a837cdca46942cda0e9548106323de7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2b6fdb197f08494da131b410570352df]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 16:37:39.356153 14997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a837cdca46942cda0e9548106323de7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:39.356153 15236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a837cdca46942cda0e9548106323de7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:39.356163 15480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a837cdca46942cda0e9548106323de7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:39.357450 15652 tablet_bootstrap.cc:492] T 3a837cdca46942cda0e9548106323de7 P fc4578a90e094dfd8f119ccd8ee2227a: Bootstrap starting.
I20260812 16:37:39.357725 15656 tablet_bootstrap.cc:492] T 3a837cdca46942cda0e9548106323de7 P 87509834f1fc44a69dc033288c16409d: Bootstrap starting.
I20260812 16:37:39.357867 15652 tablet_bootstrap.cc:654] T 3a837cdca46942cda0e9548106323de7 P fc4578a90e094dfd8f119ccd8ee2227a: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:39.358160 15656 tablet_bootstrap.cc:654] T 3a837cdca46942cda0e9548106323de7 P 87509834f1fc44a69dc033288c16409d: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:39.358542 15652 tablet_bootstrap.cc:492] T 3a837cdca46942cda0e9548106323de7 P fc4578a90e094dfd8f119ccd8ee2227a: No bootstrap required, opened a new log
I20260812 16:37:39.358556 15662 tablet_bootstrap.cc:492] T 3a837cdca46942cda0e9548106323de7 P a03e7efd0f5540a1a44160559e469cc1: Bootstrap starting.
I20260812 16:37:39.358603 15652 ts_tablet_manager.cc:1397] T 3a837cdca46942cda0e9548106323de7 P fc4578a90e094dfd8f119ccd8ee2227a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:39.358739 15652 raft_consensus.cc:348] T 3a837cdca46942cda0e9548106323de7 P fc4578a90e094dfd8f119ccd8ee2227a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } }
I20260812 16:37:39.358824 15652 raft_consensus.cc:374] T 3a837cdca46942cda0e9548106323de7 P fc4578a90e094dfd8f119ccd8ee2227a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:39.358844 15652 raft_consensus.cc:729] T 3a837cdca46942cda0e9548106323de7 P fc4578a90e094dfd8f119ccd8ee2227a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc4578a90e094dfd8f119ccd8ee2227a, State: Initialized, Role: FOLLOWER
I20260812 16:37:39.358896 15652 consensus_queue.cc:260] T 3a837cdca46942cda0e9548106323de7 P fc4578a90e094dfd8f119ccd8ee2227a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } }
I20260812 16:37:39.358964 15662 tablet_bootstrap.cc:654] T 3a837cdca46942cda0e9548106323de7 P a03e7efd0f5540a1a44160559e469cc1: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:39.359018 15652 ts_tablet_manager.cc:1428] T 3a837cdca46942cda0e9548106323de7 P fc4578a90e094dfd8f119ccd8ee2227a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:39.359122 15697 tablet_replica.cc:439] T 3a837cdca46942cda0e9548106323de7 P fc4578a90e094dfd8f119ccd8ee2227a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } } }
I20260812 16:37:39.359232 15697 tablet_replica.cc:442] T 3a837cdca46942cda0e9548106323de7 P fc4578a90e094dfd8f119ccd8ee2227a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:37:39.359494 15656 tablet_bootstrap.cc:492] T 3a837cdca46942cda0e9548106323de7 P 87509834f1fc44a69dc033288c16409d: No bootstrap required, opened a new log
I20260812 16:37:39.359551 15656 ts_tablet_manager.cc:1397] T 3a837cdca46942cda0e9548106323de7 P 87509834f1fc44a69dc033288c16409d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260812 16:37:39.359701 15656 raft_consensus.cc:348] T 3a837cdca46942cda0e9548106323de7 P 87509834f1fc44a69dc033288c16409d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } }
I20260812 16:37:39.359777 15656 raft_consensus.cc:374] T 3a837cdca46942cda0e9548106323de7 P 87509834f1fc44a69dc033288c16409d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:39.359800 15656 raft_consensus.cc:729] T 3a837cdca46942cda0e9548106323de7 P 87509834f1fc44a69dc033288c16409d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87509834f1fc44a69dc033288c16409d, State: Initialized, Role: FOLLOWER
I20260812 16:37:39.359845 15656 consensus_queue.cc:260] T 3a837cdca46942cda0e9548106323de7 P 87509834f1fc44a69dc033288c16409d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } }
I20260812 16:37:39.359954 15656 ts_tablet_manager.cc:1428] T 3a837cdca46942cda0e9548106323de7 P 87509834f1fc44a69dc033288c16409d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:39.360085 15663 tablet_replica.cc:439] T 3a837cdca46942cda0e9548106323de7 P 87509834f1fc44a69dc033288c16409d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } } }
I20260812 16:37:39.360191 15663 tablet_replica.cc:442] T 3a837cdca46942cda0e9548106323de7 P 87509834f1fc44a69dc033288c16409d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:37:39.360589 15662 tablet_bootstrap.cc:492] T 3a837cdca46942cda0e9548106323de7 P a03e7efd0f5540a1a44160559e469cc1: No bootstrap required, opened a new log
I20260812 16:37:39.360631 15662 ts_tablet_manager.cc:1397] T 3a837cdca46942cda0e9548106323de7 P a03e7efd0f5540a1a44160559e469cc1: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 16:37:39.360800 15662 raft_consensus.cc:348] T 3a837cdca46942cda0e9548106323de7 P a03e7efd0f5540a1a44160559e469cc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } }
I20260812 16:37:39.360899 15662 raft_consensus.cc:374] T 3a837cdca46942cda0e9548106323de7 P a03e7efd0f5540a1a44160559e469cc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:39.360919 15662 raft_consensus.cc:729] T 3a837cdca46942cda0e9548106323de7 P a03e7efd0f5540a1a44160559e469cc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a03e7efd0f5540a1a44160559e469cc1, State: Initialized, Role: FOLLOWER
I20260812 16:37:39.360980 15662 consensus_queue.cc:260] T 3a837cdca46942cda0e9548106323de7 P a03e7efd0f5540a1a44160559e469cc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } }
I20260812 16:37:39.361091 15662 ts_tablet_manager.cc:1428] T 3a837cdca46942cda0e9548106323de7 P a03e7efd0f5540a1a44160559e469cc1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:39.361179 15672 tablet_replica.cc:439] T 3a837cdca46942cda0e9548106323de7 P a03e7efd0f5540a1a44160559e469cc1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } } }
I20260812 16:37:39.361279 15672 tablet_replica.cc:442] T 3a837cdca46942cda0e9548106323de7 P a03e7efd0f5540a1a44160559e469cc1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:37:39.612630 15672 raft_consensus.cc:482] T 3a837cdca46942cda0e9548106323de7 P a03e7efd0f5540a1a44160559e469cc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:37:39.612746 15672 raft_consensus.cc:504] T 3a837cdca46942cda0e9548106323de7 P a03e7efd0f5540a1a44160559e469cc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } }
I20260812 16:37:39.613181 15672 leader_election.cc:288] T 3a837cdca46942cda0e9548106323de7 P a03e7efd0f5540a1a44160559e469cc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87509834f1fc44a69dc033288c16409d (127.0.253.194:33615), fc4578a90e094dfd8f119ccd8ee2227a (127.0.253.195:38573)
I20260812 16:37:39.616508 15650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:39.613472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:36867 (local address 127.0.253.195:38573)
0812 16:37:39.613524 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:39.613526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:39.613627 (+ 101us) server_negotiation.cc:407] Connection header received
0812 16:37:39.613706 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:39.613708 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:39.613719 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:39.613774 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:39.613946 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.614611 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.615757 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.616161 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.616217 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:39.616224 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:39.616382 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 16:37:39.616424 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 16:37:39.616503 15750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:39.613406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.195:38573 (local address 127.0.253.193:36867)
0812 16:37:39.613541 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:39.613552 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:39.613627 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:39.613799 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:39.613800 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:39.613803 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:39.613866 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:39.613869 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.614686 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.614687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:39.615680 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:39.615683 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.616238 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.616239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:39.616306 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:39.616311 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:37:39.616372 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 16:37:39.616404 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":5,"thread_start_us":54,"threads_started":1}
I20260812 16:37:39.616647 15752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:39.613394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:33615 (local address 127.0.253.193:50453)
0812 16:37:39.613523 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:39.613535 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:39.613664 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:39.613841 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:39.613842 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:39.613844 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:37:39.613929 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:39.613931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.614804 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.614805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:39.615741 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:39.615745 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.616322 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.616323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:39.616419 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:39.616424 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:37:39.616485 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 16:37:39.616524 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":57,"threads_started":1}
I20260812 16:37:39.616822 15753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:39.613450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:50453 (local address 127.0.253.194:33615)
0812 16:37:39.613560 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:39.613562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:39.613672 (+ 110us) server_negotiation.cc:407] Connection header received
0812 16:37:39.613735 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:39.613735 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:39.613745 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:39.613827 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:37:39.614002 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.614729 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.615840 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.616240 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.616310 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:39.616319 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:39.616504 (+ 185us) server_negotiation.cc:299] Negotiation successful
0812 16:37:39.616555 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":52,"threads_started":1}
I20260812 16:37:39.616971 15516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a837cdca46942cda0e9548106323de7" candidate_uuid: "a03e7efd0f5540a1a44160559e469cc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" is_pre_election: true
I20260812 16:37:39.617067 15516 raft_consensus.cc:2454] T 3a837cdca46942cda0e9548106323de7 P fc4578a90e094dfd8f119ccd8ee2227a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a03e7efd0f5540a1a44160559e469cc1 in term 0.
I20260812 16:37:39.617069 15268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a837cdca46942cda0e9548106323de7" candidate_uuid: "a03e7efd0f5540a1a44160559e469cc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87509834f1fc44a69dc033288c16409d" is_pre_election: true
I20260812 16:37:39.617170 15268 raft_consensus.cc:2454] T 3a837cdca46942cda0e9548106323de7 P 87509834f1fc44a69dc033288c16409d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a03e7efd0f5540a1a44160559e469cc1 in term 0.
I20260812 16:37:39.617271 14911 leader_election.cc:302] T 3a837cdca46942cda0e9548106323de7 P a03e7efd0f5540a1a44160559e469cc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a03e7efd0f5540a1a44160559e469cc1, fc4578a90e094dfd8f119ccd8ee2227a; no voters:
I20260812 16:37:39.617388 15672 raft_consensus.cc:2781] T 3a837cdca46942cda0e9548106323de7 P a03e7efd0f5540a1a44160559e469cc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:37:39.617426 15672 raft_consensus.cc:482] T 3a837cdca46942cda0e9548106323de7 P a03e7efd0f5540a1a44160559e469cc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:37:39.617455 15672 raft_consensus.cc:3037] T 3a837cdca46942cda0e9548106323de7 P a03e7efd0f5540a1a44160559e469cc1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:39.618251 15672 raft_consensus.cc:504] T 3a837cdca46942cda0e9548106323de7 P a03e7efd0f5540a1a44160559e469cc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } }
I20260812 16:37:39.618395 15672 leader_election.cc:288] T 3a837cdca46942cda0e9548106323de7 P a03e7efd0f5540a1a44160559e469cc1 [CANDIDATE]: Term 1 election: Requested vote from peers 87509834f1fc44a69dc033288c16409d (127.0.253.194:33615), fc4578a90e094dfd8f119ccd8ee2227a (127.0.253.195:38573)
I20260812 16:37:39.618520 15268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a837cdca46942cda0e9548106323de7" candidate_uuid: "a03e7efd0f5540a1a44160559e469cc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87509834f1fc44a69dc033288c16409d"
I20260812 16:37:39.618592 15268 raft_consensus.cc:3037] T 3a837cdca46942cda0e9548106323de7 P 87509834f1fc44a69dc033288c16409d [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:39.618548 15516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a837cdca46942cda0e9548106323de7" candidate_uuid: "a03e7efd0f5540a1a44160559e469cc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc4578a90e094dfd8f119ccd8ee2227a"
I20260812 16:37:39.618656 15516 raft_consensus.cc:3037] T 3a837cdca46942cda0e9548106323de7 P fc4578a90e094dfd8f119ccd8ee2227a [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:39.619292 15268 raft_consensus.cc:2454] T 3a837cdca46942cda0e9548106323de7 P 87509834f1fc44a69dc033288c16409d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a03e7efd0f5540a1a44160559e469cc1 in term 1.
I20260812 16:37:39.619442 15516 raft_consensus.cc:2454] T 3a837cdca46942cda0e9548106323de7 P fc4578a90e094dfd8f119ccd8ee2227a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a03e7efd0f5540a1a44160559e469cc1 in term 1.
I20260812 16:37:39.619460 14913 leader_election.cc:302] T 3a837cdca46942cda0e9548106323de7 P a03e7efd0f5540a1a44160559e469cc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87509834f1fc44a69dc033288c16409d, a03e7efd0f5540a1a44160559e469cc1; no voters:
I20260812 16:37:39.619591 15672 raft_consensus.cc:2781] T 3a837cdca46942cda0e9548106323de7 P a03e7efd0f5540a1a44160559e469cc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:37:39.619715 15672 raft_consensus.cc:686] T 3a837cdca46942cda0e9548106323de7 P a03e7efd0f5540a1a44160559e469cc1 [term 1 LEADER]: Becoming Leader. State: Replica: a03e7efd0f5540a1a44160559e469cc1, State: Running, Role: LEADER
I20260812 16:37:39.619786 15672 consensus_queue.cc:237] T 3a837cdca46942cda0e9548106323de7 P a03e7efd0f5540a1a44160559e469cc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } }
I20260812 16:37:39.620129 15756 tablet_replica.cc:439] T 3a837cdca46942cda0e9548106323de7 P a03e7efd0f5540a1a44160559e469cc1: TxnStatusTablet state changed. Reason: New leader a03e7efd0f5540a1a44160559e469cc1. Latest consensus state: current_term: 1 leader_uuid: "a03e7efd0f5540a1a44160559e469cc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } } }
I20260812 16:37:39.620225 15756 tablet_replica.cc:442] T 3a837cdca46942cda0e9548106323de7 P a03e7efd0f5540a1a44160559e469cc1: This TxnStatusTablet replica's current role is: LEADER
I20260812 16:37:39.620399 15761 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 16:37:39.620548 15761 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 16:37:39.620512 14753 catalog_manager.cc:5495] T 3a837cdca46942cda0e9548106323de7 P a03e7efd0f5540a1a44160559e469cc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a03e7efd0f5540a1a44160559e469cc1 (127.0.253.193). New cstate: current_term: 1 leader_uuid: "a03e7efd0f5540a1a44160559e469cc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } health_report { overall_health: UNKNOWN } } }
I20260812 16:37:39.643181 15651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:39.640438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34908 (local address 127.0.253.193:45315)
0812 16:37:39.640504 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:39.640505 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:39.640538 (+ 33us) server_negotiation.cc:407] Connection header received
0812 16:37:39.640570 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:39.640571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:39.640588 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:39.640638 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:37:39.640779 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.641228 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.642330 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.642503 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.642553 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:39.642823 (+ 270us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:37:39.642911 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:37:39.642964 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:39.643046 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 16:37:39.643063 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 16:37:39.643198 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:39.640346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:45315 (local address 127.0.0.1:34908)
0812 16:37:39.640428 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:39.640442 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:39.640495 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:39.640657 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:39.640658 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:39.640659 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 16:37:39.640713 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:39.640716 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.641301 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.641302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:39.642236 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:39.642239 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.642638 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.642639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:39.642711 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:39.642729 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 16:37:39.642976 (+ 247us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 16:37:39.642985 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 16:37:39.643042 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 16:37:39.643116 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 16:37:39.644675 15268 raft_consensus.cc:1261] T 3a837cdca46942cda0e9548106323de7 P 87509834f1fc44a69dc033288c16409d [term 1 FOLLOWER]: Refusing update from remote peer a03e7efd0f5540a1a44160559e469cc1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:37:39.644775 15516 raft_consensus.cc:1261] T 3a837cdca46942cda0e9548106323de7 P fc4578a90e094dfd8f119ccd8ee2227a [term 1 FOLLOWER]: Refusing update from remote peer a03e7efd0f5540a1a44160559e469cc1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:37:39.644959 15756 consensus_queue.cc:1035] T 3a837cdca46942cda0e9548106323de7 P a03e7efd0f5540a1a44160559e469cc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:37:39.645108 15672 consensus_queue.cc:1035] T 3a837cdca46942cda0e9548106323de7 P a03e7efd0f5540a1a44160559e469cc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:37:39.645701 15663 tablet_replica.cc:439] T 3a837cdca46942cda0e9548106323de7 P 87509834f1fc44a69dc033288c16409d: TxnStatusTablet state changed. Reason: New leader a03e7efd0f5540a1a44160559e469cc1. Latest consensus state: current_term: 1 leader_uuid: "a03e7efd0f5540a1a44160559e469cc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } } }
I20260812 16:37:39.645807 15663 tablet_replica.cc:442] T 3a837cdca46942cda0e9548106323de7 P 87509834f1fc44a69dc033288c16409d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:37:39.645787 15697 tablet_replica.cc:439] T 3a837cdca46942cda0e9548106323de7 P fc4578a90e094dfd8f119ccd8ee2227a: TxnStatusTablet state changed. Reason: New leader a03e7efd0f5540a1a44160559e469cc1. Latest consensus state: current_term: 1 leader_uuid: "a03e7efd0f5540a1a44160559e469cc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } } }
I20260812 16:37:39.645890 15697 tablet_replica.cc:442] T 3a837cdca46942cda0e9548106323de7 P fc4578a90e094dfd8f119ccd8ee2227a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:37:39.646817 15764 mvcc.cc:205] Tried to move back new op lower bound from 7317719693902422016 to 7317719693803180032. Current Snapshot: MvccSnapshot[applied={T|T < 7317719693902422016}]
I20260812 16:37:39.646955 15756 tablet_replica.cc:439] T 3a837cdca46942cda0e9548106323de7 P a03e7efd0f5540a1a44160559e469cc1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a03e7efd0f5540a1a44160559e469cc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } } }
I20260812 16:37:39.647051 15756 tablet_replica.cc:442] T 3a837cdca46942cda0e9548106323de7 P a03e7efd0f5540a1a44160559e469cc1: This TxnStatusTablet replica's current role is: LEADER
I20260812 16:37:39.647274 15756 tablet_replica.cc:439] T 3a837cdca46942cda0e9548106323de7 P a03e7efd0f5540a1a44160559e469cc1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a03e7efd0f5540a1a44160559e469cc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } } }
I20260812 16:37:39.647351 15756 tablet_replica.cc:442] T 3a837cdca46942cda0e9548106323de7 P a03e7efd0f5540a1a44160559e469cc1: This TxnStatusTablet replica's current role is: LEADER
I20260812 16:37:39.647330 15697 tablet_replica.cc:439] T 3a837cdca46942cda0e9548106323de7 P fc4578a90e094dfd8f119ccd8ee2227a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a03e7efd0f5540a1a44160559e469cc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } } }
I20260812 16:37:39.647425 15697 tablet_replica.cc:442] T 3a837cdca46942cda0e9548106323de7 P fc4578a90e094dfd8f119ccd8ee2227a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:37:39.647706 15765 mvcc.cc:205] Tried to move back new op lower bound from 7317719693902422016 to 7317719693803180032. Current Snapshot: MvccSnapshot[applied={T|T < 7317719693902422016}]
I20260812 16:37:39.647780 15663 tablet_replica.cc:439] T 3a837cdca46942cda0e9548106323de7 P 87509834f1fc44a69dc033288c16409d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a03e7efd0f5540a1a44160559e469cc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } } }
I20260812 16:37:39.647862 15663 tablet_replica.cc:442] T 3a837cdca46942cda0e9548106323de7 P 87509834f1fc44a69dc033288c16409d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:37:39.652069 15753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:39.649425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40292 (local address 127.0.253.194:33615)
0812 16:37:39.649464 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:39.649466 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:39.649528 (+ 62us) server_negotiation.cc:407] Connection header received
0812 16:37:39.649614 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:39.649615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:39.649630 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:39.649690 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:37:39.649826 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.650258 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.651379 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.651555 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.651607 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:39.651787 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:37:39.651876 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:37:39.651928 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:39.652003 (+ 75us) server_negotiation.cc:299] Negotiation successful
0812 16:37:39.652021 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 16:37:39.657302 15651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:39.654685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:58673 (local address 127.0.253.193:45315)
0812 16:37:39.654741 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:39.654743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:39.654806 (+ 63us) server_negotiation.cc:407] Connection header received
0812 16:37:39.654855 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:39.654855 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:39.654865 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:39.654928 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:37:39.655025 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.655779 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.656518 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.656832 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.656904 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:39.657073 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:37:39.657107 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:37:39.657159 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:39.657216 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 16:37:39.657234 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 16:37:39.657320 15786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:39.654572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:45315 (local address 127.0.253.194:58673)
0812 16:37:39.654742 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:39.654753 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:39.654812 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:39.654929 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:39.654929 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:39.654930 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 16:37:39.654987 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:39.654988 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.655851 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.655852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:39.656446 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:39.656447 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.656917 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.656918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:39.656980 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:39.656993 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 16:37:39.657174 (+ 181us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 16:37:39.657176 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 16:37:39.657211 (+ 35us) client_negotiation.cc:240] Negotiation successful
0812 16:37:39.657252 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":84,"threads_started":1}
I20260812 16:37:39.659373 15036 raft_consensus.cc:1261] T 825fc47862f54395bc7702747d526259 P a03e7efd0f5540a1a44160559e469cc1 [term 1 FOLLOWER]: Refusing update from remote peer 87509834f1fc44a69dc033288c16409d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:37:39.659403 15516 raft_consensus.cc:1261] T 825fc47862f54395bc7702747d526259 P fc4578a90e094dfd8f119ccd8ee2227a [term 1 FOLLOWER]: Refusing update from remote peer 87509834f1fc44a69dc033288c16409d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:37:39.659920 15719 consensus_queue.cc:1035] T 825fc47862f54395bc7702747d526259 P 87509834f1fc44a69dc033288c16409d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:37:39.660070 15663 consensus_queue.cc:1035] T 825fc47862f54395bc7702747d526259 P 87509834f1fc44a69dc033288c16409d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:37:39.668299 15753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:39.665636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:57717 (local address 127.0.253.194:33615)
0812 16:37:39.665677 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:39.665679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:39.665847 (+ 168us) server_negotiation.cc:407] Connection header received
0812 16:37:39.665907 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:39.665908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:39.665918 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:39.665957 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:37:39.666098 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.666517 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.667692 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.667865 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.667915 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:39.668119 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:37:39.668146 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:37:39.668182 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:39.668235 (+ 53us) server_negotiation.cc:299] Negotiation successful
0812 16:37:39.668252 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 16:37:39.668370 15800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:39.665539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:33615 (local address 127.0.253.193:57717)
0812 16:37:39.665781 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:39.665792 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:39.665851 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:39.665970 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:39.665971 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:39.665972 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 16:37:39.666042 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:39.666043 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.666590 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.666591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:39.667599 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:39.667601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.667938 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.667939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:39.668022 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:39.668039 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 16:37:39.668183 (+ 144us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 16:37:39.668185 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 16:37:39.668233 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 16:37:39.668273 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":71,"threads_started":1}
I20260812 16:37:39.677547 14747 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:37:39.677659 14747 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:37:39.678354 14747 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d520c8ddc0ff4a1e9d90dfab76998e47: Sending DeleteTablet for 3 replicas of tablet 825fc47862f54395bc7702747d526259
I20260812 16:37:39.678604 15480 tablet_service.cc:1514] Processing DeleteTablet for tablet 825fc47862f54395bc7702747d526259 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:39 UTC) from {username='oryx', principal='oryx/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:48312
I20260812 16:37:39.678668 15236 tablet_service.cc:1514] Processing DeleteTablet for tablet 825fc47862f54395bc7702747d526259 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:39 UTC) from {username='oryx', principal='oryx/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:40278
I20260812 16:37:39.678874 15806 tablet_replica.cc:330] T 825fc47862f54395bc7702747d526259 P 87509834f1fc44a69dc033288c16409d: stopping tablet replica
I20260812 16:37:39.678936 15806 raft_consensus.cc:2229] T 825fc47862f54395bc7702747d526259 P 87509834f1fc44a69dc033288c16409d [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:37:39.679019 15806 raft_consensus.cc:2258] T 825fc47862f54395bc7702747d526259 P 87509834f1fc44a69dc033288c16409d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:39.679010 14997 tablet_service.cc:1514] Processing DeleteTablet for tablet 825fc47862f54395bc7702747d526259 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:39 UTC) from {username='oryx', principal='oryx/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:34898
I20260812 16:37:39.679232 15805 tablet_replica.cc:330] T 825fc47862f54395bc7702747d526259 P fc4578a90e094dfd8f119ccd8ee2227a: stopping tablet replica
I20260812 16:37:39.679286 15805 raft_consensus.cc:2229] T 825fc47862f54395bc7702747d526259 P fc4578a90e094dfd8f119ccd8ee2227a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:37:39.679328 15805 raft_consensus.cc:2258] T 825fc47862f54395bc7702747d526259 P fc4578a90e094dfd8f119ccd8ee2227a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:39.679530 15806 ts_tablet_manager.cc:1905] T 825fc47862f54395bc7702747d526259 P 87509834f1fc44a69dc033288c16409d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:37:39.679582 15805 ts_tablet_manager.cc:1905] T 825fc47862f54395bc7702747d526259 P fc4578a90e094dfd8f119ccd8ee2227a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:37:39.681492 15805 ts_tablet_manager.cc:1918] T 825fc47862f54395bc7702747d526259 P fc4578a90e094dfd8f119ccd8ee2227a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:37:39.681561 15805 log.cc:1196] T 825fc47862f54395bc7702747d526259 P fc4578a90e094dfd8f119ccd8ee2227a: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-2/wal/wals/825fc47862f54395bc7702747d526259
I20260812 16:37:39.681874 15805 ts_tablet_manager.cc:1939] T 825fc47862f54395bc7702747d526259 P fc4578a90e094dfd8f119ccd8ee2227a: Deleting consensus metadata
I20260812 16:37:39.682185 14718 catalog_manager.cc:4841] TS fc4578a90e094dfd8f119ccd8ee2227a (127.0.253.195:38573): tablet 825fc47862f54395bc7702747d526259 (table test-table [id=2715a40656434621aee6a29331e7e2e4]) successfully deleted
I20260812 16:37:39.685030 15806 ts_tablet_manager.cc:1918] T 825fc47862f54395bc7702747d526259 P 87509834f1fc44a69dc033288c16409d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:37:39.685089 15806 log.cc:1196] T 825fc47862f54395bc7702747d526259 P 87509834f1fc44a69dc033288c16409d: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-1/wal/wals/825fc47862f54395bc7702747d526259
I20260812 16:37:39.685340 15806 ts_tablet_manager.cc:1939] T 825fc47862f54395bc7702747d526259 P 87509834f1fc44a69dc033288c16409d: Deleting consensus metadata
I20260812 16:37:39.685843 14721 catalog_manager.cc:4841] TS 87509834f1fc44a69dc033288c16409d (127.0.253.194:33615): tablet 825fc47862f54395bc7702747d526259 (table test-table [id=2715a40656434621aee6a29331e7e2e4]) successfully deleted
I20260812 16:37:39.685887 15810 tablet_replica.cc:330] T 825fc47862f54395bc7702747d526259 P a03e7efd0f5540a1a44160559e469cc1: stopping tablet replica
I20260812 16:37:39.686002 15810 raft_consensus.cc:2229] T 825fc47862f54395bc7702747d526259 P a03e7efd0f5540a1a44160559e469cc1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:37:39.686070 15810 raft_consensus.cc:2258] T 825fc47862f54395bc7702747d526259 P a03e7efd0f5540a1a44160559e469cc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:39.687508 15810 ts_tablet_manager.cc:1905] T 825fc47862f54395bc7702747d526259 P a03e7efd0f5540a1a44160559e469cc1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:37:39.689244 15810 ts_tablet_manager.cc:1918] T 825fc47862f54395bc7702747d526259 P a03e7efd0f5540a1a44160559e469cc1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:37:39.689338 15810 log.cc:1196] T 825fc47862f54395bc7702747d526259 P a03e7efd0f5540a1a44160559e469cc1: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-0/wal/wals/825fc47862f54395bc7702747d526259
I20260812 16:37:39.689586 15810 ts_tablet_manager.cc:1939] T 825fc47862f54395bc7702747d526259 P a03e7efd0f5540a1a44160559e469cc1: Deleting consensus metadata
I20260812 16:37:39.689915 14721 catalog_manager.cc:4841] TS a03e7efd0f5540a1a44160559e469cc1 (127.0.253.193:45315): tablet 825fc47862f54395bc7702747d526259 (table test-table [id=2715a40656434621aee6a29331e7e2e4]) successfully deleted
Aug 12 16:37:39 dist-test-slave-w4v5 krb5kdc[14663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 16:37:39 dist-test-slave-w4v5 krb5kdc[14663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:39 dist-test-slave-w4v5 krb5kdc[14663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
W20260812 16:37:39.715305 15741 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:39.708148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35260 (local address 127.0.253.254:33909)
0812 16:37:39.708204 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:39.708208 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:39.708350 (+ 142us) server_negotiation.cc:407] Connection header received
0812 16:37:39.708413 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:39.708415 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:39.708434 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:39.708501 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:39.709229 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.710033 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.711189 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.711504 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.711579 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:39.715230 (+ 3651us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35260: BlockingRecv error: recv got EOF from 127.0.0.1:35260 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 12 16:37:39 dist-test-slave-w4v5 krb5kdc[14663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM
I20260812 16:37:39.725258 15741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:39.716115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35274 (local address 127.0.253.254:33909)
0812 16:37:39.716162 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:39.716163 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:39.716480 (+ 317us) server_negotiation.cc:407] Connection header received
0812 16:37:39.716541 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:39.716542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:39.716552 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:39.716614 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:39.717225 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.717970 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.718993 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.719351 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.719419 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:39.722151 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:39.722153 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:39.722153 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:39.722161 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:39.723976 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:39.724399 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:39.724400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:39.724401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:39.724469 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:39.724697 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:39.724699 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:39.724699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:39.724846 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:39.724964 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:39.725076 (+ 112us) server_negotiation.cc:299] Negotiation successful
0812 16:37:39.725163 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 16:37:39.727216 14747 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35274:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:37:39.727408 14747 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:37:39.728902 15480 tablet_service.cc:1467] Processing CreateTablet for tablet 796ae3570fa741fb8677572a4e39da54 (DEFAULT_TABLE table=test-table [id=e23b11bb94ea46669ac4af01ae31638a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:39.728956 14997 tablet_service.cc:1467] Processing CreateTablet for tablet 796ae3570fa741fb8677572a4e39da54 (DEFAULT_TABLE table=test-table [id=e23b11bb94ea46669ac4af01ae31638a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:39.729051 15480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 796ae3570fa741fb8677572a4e39da54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:39.729099 14997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 796ae3570fa741fb8677572a4e39da54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:39.730410 15652 tablet_bootstrap.cc:492] T 796ae3570fa741fb8677572a4e39da54 P fc4578a90e094dfd8f119ccd8ee2227a: Bootstrap starting.
I20260812 16:37:39.730581 15236 tablet_service.cc:1467] Processing CreateTablet for tablet 796ae3570fa741fb8677572a4e39da54 (DEFAULT_TABLE table=test-table [id=e23b11bb94ea46669ac4af01ae31638a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:39.730710 15236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 796ae3570fa741fb8677572a4e39da54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:39.730805 15652 tablet_bootstrap.cc:654] T 796ae3570fa741fb8677572a4e39da54 P fc4578a90e094dfd8f119ccd8ee2227a: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:39.731151 15662 tablet_bootstrap.cc:492] T 796ae3570fa741fb8677572a4e39da54 P a03e7efd0f5540a1a44160559e469cc1: Bootstrap starting.
I20260812 16:37:39.731590 15662 tablet_bootstrap.cc:654] T 796ae3570fa741fb8677572a4e39da54 P a03e7efd0f5540a1a44160559e469cc1: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:39.731865 15652 tablet_bootstrap.cc:492] T 796ae3570fa741fb8677572a4e39da54 P fc4578a90e094dfd8f119ccd8ee2227a: No bootstrap required, opened a new log
I20260812 16:37:39.731909 15652 ts_tablet_manager.cc:1397] T 796ae3570fa741fb8677572a4e39da54 P fc4578a90e094dfd8f119ccd8ee2227a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:37:39.732053 15652 raft_consensus.cc:348] T 796ae3570fa741fb8677572a4e39da54 P fc4578a90e094dfd8f119ccd8ee2227a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } }
I20260812 16:37:39.732128 15652 raft_consensus.cc:374] T 796ae3570fa741fb8677572a4e39da54 P fc4578a90e094dfd8f119ccd8ee2227a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:39.732165 15652 raft_consensus.cc:729] T 796ae3570fa741fb8677572a4e39da54 P fc4578a90e094dfd8f119ccd8ee2227a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc4578a90e094dfd8f119ccd8ee2227a, State: Initialized, Role: FOLLOWER
I20260812 16:37:39.732208 15652 consensus_queue.cc:260] T 796ae3570fa741fb8677572a4e39da54 P fc4578a90e094dfd8f119ccd8ee2227a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } }
I20260812 16:37:39.732293 15652 ts_tablet_manager.cc:1428] T 796ae3570fa741fb8677572a4e39da54 P fc4578a90e094dfd8f119ccd8ee2227a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 16:37:39.733451 15656 tablet_bootstrap.cc:492] T 796ae3570fa741fb8677572a4e39da54 P 87509834f1fc44a69dc033288c16409d: Bootstrap starting.
I20260812 16:37:39.733916 15656 tablet_bootstrap.cc:654] T 796ae3570fa741fb8677572a4e39da54 P 87509834f1fc44a69dc033288c16409d: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:39.734043 15662 tablet_bootstrap.cc:492] T 796ae3570fa741fb8677572a4e39da54 P a03e7efd0f5540a1a44160559e469cc1: No bootstrap required, opened a new log
I20260812 16:37:39.734091 15662 ts_tablet_manager.cc:1397] T 796ae3570fa741fb8677572a4e39da54 P a03e7efd0f5540a1a44160559e469cc1: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260812 16:37:39.734282 15662 raft_consensus.cc:348] T 796ae3570fa741fb8677572a4e39da54 P a03e7efd0f5540a1a44160559e469cc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } }
I20260812 16:37:39.734364 15662 raft_consensus.cc:374] T 796ae3570fa741fb8677572a4e39da54 P a03e7efd0f5540a1a44160559e469cc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:39.734386 15662 raft_consensus.cc:729] T 796ae3570fa741fb8677572a4e39da54 P a03e7efd0f5540a1a44160559e469cc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a03e7efd0f5540a1a44160559e469cc1, State: Initialized, Role: FOLLOWER
I20260812 16:37:39.734438 15662 consensus_queue.cc:260] T 796ae3570fa741fb8677572a4e39da54 P a03e7efd0f5540a1a44160559e469cc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } }
I20260812 16:37:39.734529 15662 ts_tablet_manager.cc:1428] T 796ae3570fa741fb8677572a4e39da54 P a03e7efd0f5540a1a44160559e469cc1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:39.735174 15656 tablet_bootstrap.cc:492] T 796ae3570fa741fb8677572a4e39da54 P 87509834f1fc44a69dc033288c16409d: No bootstrap required, opened a new log
I20260812 16:37:39.735227 15656 ts_tablet_manager.cc:1397] T 796ae3570fa741fb8677572a4e39da54 P 87509834f1fc44a69dc033288c16409d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 16:37:39.735363 15656 raft_consensus.cc:348] T 796ae3570fa741fb8677572a4e39da54 P 87509834f1fc44a69dc033288c16409d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } }
I20260812 16:37:39.735424 15656 raft_consensus.cc:374] T 796ae3570fa741fb8677572a4e39da54 P 87509834f1fc44a69dc033288c16409d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:39.735445 15656 raft_consensus.cc:729] T 796ae3570fa741fb8677572a4e39da54 P 87509834f1fc44a69dc033288c16409d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87509834f1fc44a69dc033288c16409d, State: Initialized, Role: FOLLOWER
I20260812 16:37:39.735494 15656 consensus_queue.cc:260] T 796ae3570fa741fb8677572a4e39da54 P 87509834f1fc44a69dc033288c16409d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } }
I20260812 16:37:39.735594 15656 ts_tablet_manager.cc:1428] T 796ae3570fa741fb8677572a4e39da54 P 87509834f1fc44a69dc033288c16409d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:39.800354 15697 raft_consensus.cc:482] T 796ae3570fa741fb8677572a4e39da54 P fc4578a90e094dfd8f119ccd8ee2227a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:37:39.800457 15697 raft_consensus.cc:504] T 796ae3570fa741fb8677572a4e39da54 P fc4578a90e094dfd8f119ccd8ee2227a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } }
I20260812 16:37:39.800761 15697 leader_election.cc:288] T 796ae3570fa741fb8677572a4e39da54 P fc4578a90e094dfd8f119ccd8ee2227a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a03e7efd0f5540a1a44160559e469cc1 (127.0.253.193:45315), 87509834f1fc44a69dc033288c16409d (127.0.253.194:33615)
I20260812 16:37:39.801004 15036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "796ae3570fa741fb8677572a4e39da54" candidate_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a03e7efd0f5540a1a44160559e469cc1" is_pre_election: true
I20260812 16:37:39.801031 15268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "796ae3570fa741fb8677572a4e39da54" candidate_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87509834f1fc44a69dc033288c16409d" is_pre_election: true
I20260812 16:37:39.801116 15036 raft_consensus.cc:2454] T 796ae3570fa741fb8677572a4e39da54 P a03e7efd0f5540a1a44160559e469cc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc4578a90e094dfd8f119ccd8ee2227a in term 0.
I20260812 16:37:39.801141 15268 raft_consensus.cc:2454] T 796ae3570fa741fb8677572a4e39da54 P 87509834f1fc44a69dc033288c16409d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc4578a90e094dfd8f119ccd8ee2227a in term 0.
I20260812 16:37:39.801280 15401 leader_election.cc:302] T 796ae3570fa741fb8677572a4e39da54 P fc4578a90e094dfd8f119ccd8ee2227a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a03e7efd0f5540a1a44160559e469cc1, fc4578a90e094dfd8f119ccd8ee2227a; no voters:
I20260812 16:37:39.801383 15697 raft_consensus.cc:2781] T 796ae3570fa741fb8677572a4e39da54 P fc4578a90e094dfd8f119ccd8ee2227a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:37:39.801442 15697 raft_consensus.cc:482] T 796ae3570fa741fb8677572a4e39da54 P fc4578a90e094dfd8f119ccd8ee2227a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:37:39.801471 15697 raft_consensus.cc:3037] T 796ae3570fa741fb8677572a4e39da54 P fc4578a90e094dfd8f119ccd8ee2227a [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:39.802263 15697 raft_consensus.cc:504] T 796ae3570fa741fb8677572a4e39da54 P fc4578a90e094dfd8f119ccd8ee2227a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } }
I20260812 16:37:39.802404 15697 leader_election.cc:288] T 796ae3570fa741fb8677572a4e39da54 P fc4578a90e094dfd8f119ccd8ee2227a [CANDIDATE]: Term 1 election: Requested vote from peers a03e7efd0f5540a1a44160559e469cc1 (127.0.253.193:45315), 87509834f1fc44a69dc033288c16409d (127.0.253.194:33615)
I20260812 16:37:39.802547 15036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "796ae3570fa741fb8677572a4e39da54" candidate_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a03e7efd0f5540a1a44160559e469cc1"
I20260812 16:37:39.802632 15036 raft_consensus.cc:3037] T 796ae3570fa741fb8677572a4e39da54 P a03e7efd0f5540a1a44160559e469cc1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:39.802609 15268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "796ae3570fa741fb8677572a4e39da54" candidate_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87509834f1fc44a69dc033288c16409d"
I20260812 16:37:39.802709 15268 raft_consensus.cc:3037] T 796ae3570fa741fb8677572a4e39da54 P 87509834f1fc44a69dc033288c16409d [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:39.803462 15268 raft_consensus.cc:2454] T 796ae3570fa741fb8677572a4e39da54 P 87509834f1fc44a69dc033288c16409d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc4578a90e094dfd8f119ccd8ee2227a in term 1.
I20260812 16:37:39.803540 15036 raft_consensus.cc:2454] T 796ae3570fa741fb8677572a4e39da54 P a03e7efd0f5540a1a44160559e469cc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc4578a90e094dfd8f119ccd8ee2227a in term 1.
I20260812 16:37:39.803663 15401 leader_election.cc:302] T 796ae3570fa741fb8677572a4e39da54 P fc4578a90e094dfd8f119ccd8ee2227a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87509834f1fc44a69dc033288c16409d, fc4578a90e094dfd8f119ccd8ee2227a; no voters:
I20260812 16:37:39.803825 15697 raft_consensus.cc:2781] T 796ae3570fa741fb8677572a4e39da54 P fc4578a90e094dfd8f119ccd8ee2227a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:37:39.804023 15697 raft_consensus.cc:686] T 796ae3570fa741fb8677572a4e39da54 P fc4578a90e094dfd8f119ccd8ee2227a [term 1 LEADER]: Becoming Leader. State: Replica: fc4578a90e094dfd8f119ccd8ee2227a, State: Running, Role: LEADER
I20260812 16:37:39.804090 15697 consensus_queue.cc:237] T 796ae3570fa741fb8677572a4e39da54 P fc4578a90e094dfd8f119ccd8ee2227a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } }
I20260812 16:37:39.804795 14747 catalog_manager.cc:5495] T 796ae3570fa741fb8677572a4e39da54 P fc4578a90e094dfd8f119ccd8ee2227a reported cstate change: term changed from 0 to 1, leader changed from <none> to fc4578a90e094dfd8f119ccd8ee2227a (127.0.253.195). New cstate: current_term: 1 leader_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } health_report { overall_health: HEALTHY } } }
I20260812 16:37:39.812145 15650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:39.809645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48330 (local address 127.0.253.195:38573)
0812 16:37:39.809686 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:39.809688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:39.809697 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:37:39.809747 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:39.809748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:39.809756 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:39.809802 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:37:39.809912 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.810377 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.811547 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.811734 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.811787 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:39.811935 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:37:39.811962 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:37:39.812004 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:39.812072 (+ 68us) server_negotiation.cc:299] Negotiation successful
0812 16:37:39.812091 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 16:37:39.813150 15268 raft_consensus.cc:1261] T 796ae3570fa741fb8677572a4e39da54 P 87509834f1fc44a69dc033288c16409d [term 1 FOLLOWER]: Refusing update from remote peer fc4578a90e094dfd8f119ccd8ee2227a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:37:39.813215 15036 raft_consensus.cc:1261] T 796ae3570fa741fb8677572a4e39da54 P a03e7efd0f5540a1a44160559e469cc1 [term 1 FOLLOWER]: Refusing update from remote peer fc4578a90e094dfd8f119ccd8ee2227a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:37:39.813393 15697 consensus_queue.cc:1035] T 796ae3570fa741fb8677572a4e39da54 P fc4578a90e094dfd8f119ccd8ee2227a [LEADER]: Connected to new peer: Peer: permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:37:39.813580 15837 consensus_queue.cc:1035] T 796ae3570fa741fb8677572a4e39da54 P fc4578a90e094dfd8f119ccd8ee2227a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:37:39.816838 14751 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35274:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:37:39.816943 14751 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35274:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:37:39.817706 14751 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d520c8ddc0ff4a1e9d90dfab76998e47: Sending DeleteTablet for 3 replicas of tablet 796ae3570fa741fb8677572a4e39da54
I20260812 16:37:39.817986 14997 tablet_service.cc:1514] Processing DeleteTablet for tablet 796ae3570fa741fb8677572a4e39da54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:39 UTC) from {username='oryx', principal='oryx/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:34898
I20260812 16:37:39.818005 15480 tablet_service.cc:1514] Processing DeleteTablet for tablet 796ae3570fa741fb8677572a4e39da54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:39 UTC) from {username='oryx', principal='oryx/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:48312
I20260812 16:37:39.818095 15805 tablet_replica.cc:330] T 796ae3570fa741fb8677572a4e39da54 P fc4578a90e094dfd8f119ccd8ee2227a: stopping tablet replica
I20260812 16:37:39.818099 15810 tablet_replica.cc:330] T 796ae3570fa741fb8677572a4e39da54 P a03e7efd0f5540a1a44160559e469cc1: stopping tablet replica
I20260812 16:37:39.818154 15805 raft_consensus.cc:2229] T 796ae3570fa741fb8677572a4e39da54 P fc4578a90e094dfd8f119ccd8ee2227a [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:37:39.818176 15810 raft_consensus.cc:2229] T 796ae3570fa741fb8677572a4e39da54 P a03e7efd0f5540a1a44160559e469cc1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:37:39.818159 15236 tablet_service.cc:1514] Processing DeleteTablet for tablet 796ae3570fa741fb8677572a4e39da54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:39 UTC) from {username='oryx', principal='oryx/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:40278
I20260812 16:37:39.818216 15805 raft_consensus.cc:2258] T 796ae3570fa741fb8677572a4e39da54 P fc4578a90e094dfd8f119ccd8ee2227a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:39.818222 15810 raft_consensus.cc:2258] T 796ae3570fa741fb8677572a4e39da54 P a03e7efd0f5540a1a44160559e469cc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:39.818238 15806 tablet_replica.cc:330] T 796ae3570fa741fb8677572a4e39da54 P 87509834f1fc44a69dc033288c16409d: stopping tablet replica
I20260812 16:37:39.818287 15806 raft_consensus.cc:2229] T 796ae3570fa741fb8677572a4e39da54 P 87509834f1fc44a69dc033288c16409d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:37:39.818331 15806 raft_consensus.cc:2258] T 796ae3570fa741fb8677572a4e39da54 P 87509834f1fc44a69dc033288c16409d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:39.818502 15805 ts_tablet_manager.cc:1905] T 796ae3570fa741fb8677572a4e39da54 P fc4578a90e094dfd8f119ccd8ee2227a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:37:39.818605 14751 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35274:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:37:39.818768 14751 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:37:39.818792 15806 ts_tablet_manager.cc:1905] T 796ae3570fa741fb8677572a4e39da54 P 87509834f1fc44a69dc033288c16409d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:37:39.820017 15480 tablet_service.cc:1467] Processing CreateTablet for tablet f0f7177c76ec408ab20f70d30f6836cd (DEFAULT_TABLE table=test-table [id=68887d8649ce4282a20cbb5165b8f5ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:39.820127 15480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0f7177c76ec408ab20f70d30f6836cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:39.820363 15805 ts_tablet_manager.cc:1918] T 796ae3570fa741fb8677572a4e39da54 P fc4578a90e094dfd8f119ccd8ee2227a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:37:39.820429 15805 log.cc:1196] T 796ae3570fa741fb8677572a4e39da54 P fc4578a90e094dfd8f119ccd8ee2227a: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-2/wal/wals/796ae3570fa741fb8677572a4e39da54
I20260812 16:37:39.820482 15806 ts_tablet_manager.cc:1918] T 796ae3570fa741fb8677572a4e39da54 P 87509834f1fc44a69dc033288c16409d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:37:39.820544 15806 log.cc:1196] T 796ae3570fa741fb8677572a4e39da54 P 87509834f1fc44a69dc033288c16409d: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-1/wal/wals/796ae3570fa741fb8677572a4e39da54
I20260812 16:37:39.820669 15805 ts_tablet_manager.cc:1939] T 796ae3570fa741fb8677572a4e39da54 P fc4578a90e094dfd8f119ccd8ee2227a: Deleting consensus metadata
I20260812 16:37:39.820801 15806 ts_tablet_manager.cc:1939] T 796ae3570fa741fb8677572a4e39da54 P 87509834f1fc44a69dc033288c16409d: Deleting consensus metadata
I20260812 16:37:39.820991 14718 catalog_manager.cc:4841] TS fc4578a90e094dfd8f119ccd8ee2227a (127.0.253.195:38573): tablet 796ae3570fa741fb8677572a4e39da54 (table test-table [id=e23b11bb94ea46669ac4af01ae31638a]) successfully deleted
I20260812 16:37:39.821110 14997 tablet_service.cc:1467] Processing CreateTablet for tablet f0f7177c76ec408ab20f70d30f6836cd (DEFAULT_TABLE table=test-table [id=68887d8649ce4282a20cbb5165b8f5ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:39.821234 14997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0f7177c76ec408ab20f70d30f6836cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:39.821612 15652 tablet_bootstrap.cc:492] T f0f7177c76ec408ab20f70d30f6836cd P fc4578a90e094dfd8f119ccd8ee2227a: Bootstrap starting.
I20260812 16:37:39.821920 15652 tablet_bootstrap.cc:654] T f0f7177c76ec408ab20f70d30f6836cd P fc4578a90e094dfd8f119ccd8ee2227a: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:39.822520 15652 tablet_bootstrap.cc:492] T f0f7177c76ec408ab20f70d30f6836cd P fc4578a90e094dfd8f119ccd8ee2227a: No bootstrap required, opened a new log
I20260812 16:37:39.822588 15652 ts_tablet_manager.cc:1397] T f0f7177c76ec408ab20f70d30f6836cd P fc4578a90e094dfd8f119ccd8ee2227a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:39.822629 15662 tablet_bootstrap.cc:492] T f0f7177c76ec408ab20f70d30f6836cd P a03e7efd0f5540a1a44160559e469cc1: Bootstrap starting.
I20260812 16:37:39.822721 15652 raft_consensus.cc:348] T f0f7177c76ec408ab20f70d30f6836cd P fc4578a90e094dfd8f119ccd8ee2227a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } }
I20260812 16:37:39.822782 15810 ts_tablet_manager.cc:1905] T 796ae3570fa741fb8677572a4e39da54 P a03e7efd0f5540a1a44160559e469cc1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:37:39.822798 15652 raft_consensus.cc:374] T f0f7177c76ec408ab20f70d30f6836cd P fc4578a90e094dfd8f119ccd8ee2227a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:39.822821 15652 raft_consensus.cc:729] T f0f7177c76ec408ab20f70d30f6836cd P fc4578a90e094dfd8f119ccd8ee2227a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc4578a90e094dfd8f119ccd8ee2227a, State: Initialized, Role: FOLLOWER
I20260812 16:37:39.822870 15652 consensus_queue.cc:260] T f0f7177c76ec408ab20f70d30f6836cd P fc4578a90e094dfd8f119ccd8ee2227a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } }
I20260812 16:37:39.822991 15652 ts_tablet_manager.cc:1428] T f0f7177c76ec408ab20f70d30f6836cd P fc4578a90e094dfd8f119ccd8ee2227a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 16:37:39.823001 15662 tablet_bootstrap.cc:654] T f0f7177c76ec408ab20f70d30f6836cd P a03e7efd0f5540a1a44160559e469cc1: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:39.823343 14721 catalog_manager.cc:4841] TS 87509834f1fc44a69dc033288c16409d (127.0.253.194:33615): tablet 796ae3570fa741fb8677572a4e39da54 (table test-table [id=e23b11bb94ea46669ac4af01ae31638a]) successfully deleted
I20260812 16:37:39.823359 15236 tablet_service.cc:1467] Processing CreateTablet for tablet f0f7177c76ec408ab20f70d30f6836cd (DEFAULT_TABLE table=test-table [id=68887d8649ce4282a20cbb5165b8f5ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:37:39.823462 15236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0f7177c76ec408ab20f70d30f6836cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:39.824498 15656 tablet_bootstrap.cc:492] T f0f7177c76ec408ab20f70d30f6836cd P 87509834f1fc44a69dc033288c16409d: Bootstrap starting.
I20260812 16:37:39.824627 15810 ts_tablet_manager.cc:1918] T 796ae3570fa741fb8677572a4e39da54 P a03e7efd0f5540a1a44160559e469cc1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:37:39.824752 15810 log.cc:1196] T 796ae3570fa741fb8677572a4e39da54 P a03e7efd0f5540a1a44160559e469cc1: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-0/wal/wals/796ae3570fa741fb8677572a4e39da54
I20260812 16:37:39.824827 15656 tablet_bootstrap.cc:654] T f0f7177c76ec408ab20f70d30f6836cd P 87509834f1fc44a69dc033288c16409d: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:39.825013 15810 ts_tablet_manager.cc:1939] T 796ae3570fa741fb8677572a4e39da54 P a03e7efd0f5540a1a44160559e469cc1: Deleting consensus metadata
I20260812 16:37:39.825212 15662 tablet_bootstrap.cc:492] T f0f7177c76ec408ab20f70d30f6836cd P a03e7efd0f5540a1a44160559e469cc1: No bootstrap required, opened a new log
I20260812 16:37:39.825264 15662 ts_tablet_manager.cc:1397] T f0f7177c76ec408ab20f70d30f6836cd P a03e7efd0f5540a1a44160559e469cc1: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 16:37:39.825410 15656 tablet_bootstrap.cc:492] T f0f7177c76ec408ab20f70d30f6836cd P 87509834f1fc44a69dc033288c16409d: No bootstrap required, opened a new log
I20260812 16:37:39.825383 15662 raft_consensus.cc:348] T f0f7177c76ec408ab20f70d30f6836cd P a03e7efd0f5540a1a44160559e469cc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } }
I20260812 16:37:39.825441 15662 raft_consensus.cc:374] T f0f7177c76ec408ab20f70d30f6836cd P a03e7efd0f5540a1a44160559e469cc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:39.825444 15656 ts_tablet_manager.cc:1397] T f0f7177c76ec408ab20f70d30f6836cd P 87509834f1fc44a69dc033288c16409d: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 16:37:39.825455 15662 raft_consensus.cc:729] T f0f7177c76ec408ab20f70d30f6836cd P a03e7efd0f5540a1a44160559e469cc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a03e7efd0f5540a1a44160559e469cc1, State: Initialized, Role: FOLLOWER
I20260812 16:37:39.825477 15662 consensus_queue.cc:260] T f0f7177c76ec408ab20f70d30f6836cd P a03e7efd0f5540a1a44160559e469cc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } }
I20260812 16:37:39.825546 15662 ts_tablet_manager.cc:1428] T f0f7177c76ec408ab20f70d30f6836cd P a03e7efd0f5540a1a44160559e469cc1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:39.825522 15656 raft_consensus.cc:348] T f0f7177c76ec408ab20f70d30f6836cd P 87509834f1fc44a69dc033288c16409d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } }
I20260812 16:37:39.825631 15656 raft_consensus.cc:374] T f0f7177c76ec408ab20f70d30f6836cd P 87509834f1fc44a69dc033288c16409d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:39.825657 15656 raft_consensus.cc:729] T f0f7177c76ec408ab20f70d30f6836cd P 87509834f1fc44a69dc033288c16409d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87509834f1fc44a69dc033288c16409d, State: Initialized, Role: FOLLOWER
I20260812 16:37:39.825702 15656 consensus_queue.cc:260] T f0f7177c76ec408ab20f70d30f6836cd P 87509834f1fc44a69dc033288c16409d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } }
I20260812 16:37:39.825708 14721 catalog_manager.cc:4841] TS a03e7efd0f5540a1a44160559e469cc1 (127.0.253.193:45315): tablet 796ae3570fa741fb8677572a4e39da54 (table test-table [id=e23b11bb94ea46669ac4af01ae31638a]) successfully deleted
I20260812 16:37:39.825825 15656 ts_tablet_manager.cc:1428] T f0f7177c76ec408ab20f70d30f6836cd P 87509834f1fc44a69dc033288c16409d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:39.849952 15773 raft_consensus.cc:482] T f0f7177c76ec408ab20f70d30f6836cd P a03e7efd0f5540a1a44160559e469cc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:37:39.850092 15773 raft_consensus.cc:504] T f0f7177c76ec408ab20f70d30f6836cd P a03e7efd0f5540a1a44160559e469cc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } }
I20260812 16:37:39.850291 15773 leader_election.cc:288] T f0f7177c76ec408ab20f70d30f6836cd P a03e7efd0f5540a1a44160559e469cc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87509834f1fc44a69dc033288c16409d (127.0.253.194:33615), fc4578a90e094dfd8f119ccd8ee2227a (127.0.253.195:38573)
I20260812 16:37:39.850521 15516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0f7177c76ec408ab20f70d30f6836cd" candidate_uuid: "a03e7efd0f5540a1a44160559e469cc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" is_pre_election: true
I20260812 16:37:39.850569 15268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0f7177c76ec408ab20f70d30f6836cd" candidate_uuid: "a03e7efd0f5540a1a44160559e469cc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87509834f1fc44a69dc033288c16409d" is_pre_election: true
I20260812 16:37:39.850670 15516 raft_consensus.cc:2454] T f0f7177c76ec408ab20f70d30f6836cd P fc4578a90e094dfd8f119ccd8ee2227a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a03e7efd0f5540a1a44160559e469cc1 in term 0.
I20260812 16:37:39.850715 15268 raft_consensus.cc:2454] T f0f7177c76ec408ab20f70d30f6836cd P 87509834f1fc44a69dc033288c16409d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a03e7efd0f5540a1a44160559e469cc1 in term 0.
I20260812 16:37:39.850867 14911 leader_election.cc:302] T f0f7177c76ec408ab20f70d30f6836cd P a03e7efd0f5540a1a44160559e469cc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a03e7efd0f5540a1a44160559e469cc1, fc4578a90e094dfd8f119ccd8ee2227a; no voters:
I20260812 16:37:39.850980 15773 raft_consensus.cc:2781] T f0f7177c76ec408ab20f70d30f6836cd P a03e7efd0f5540a1a44160559e469cc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:37:39.851032 15773 raft_consensus.cc:482] T f0f7177c76ec408ab20f70d30f6836cd P a03e7efd0f5540a1a44160559e469cc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:37:39.851063 15773 raft_consensus.cc:3037] T f0f7177c76ec408ab20f70d30f6836cd P a03e7efd0f5540a1a44160559e469cc1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:39.851830 15773 raft_consensus.cc:504] T f0f7177c76ec408ab20f70d30f6836cd P a03e7efd0f5540a1a44160559e469cc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } }
I20260812 16:37:39.851980 15773 leader_election.cc:288] T f0f7177c76ec408ab20f70d30f6836cd P a03e7efd0f5540a1a44160559e469cc1 [CANDIDATE]: Term 1 election: Requested vote from peers 87509834f1fc44a69dc033288c16409d (127.0.253.194:33615), fc4578a90e094dfd8f119ccd8ee2227a (127.0.253.195:38573)
I20260812 16:37:39.852172 15516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0f7177c76ec408ab20f70d30f6836cd" candidate_uuid: "a03e7efd0f5540a1a44160559e469cc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc4578a90e094dfd8f119ccd8ee2227a"
I20260812 16:37:39.852260 15516 raft_consensus.cc:3037] T f0f7177c76ec408ab20f70d30f6836cd P fc4578a90e094dfd8f119ccd8ee2227a [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:39.852226 15268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0f7177c76ec408ab20f70d30f6836cd" candidate_uuid: "a03e7efd0f5540a1a44160559e469cc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87509834f1fc44a69dc033288c16409d"
I20260812 16:37:39.852325 15268 raft_consensus.cc:3037] T f0f7177c76ec408ab20f70d30f6836cd P 87509834f1fc44a69dc033288c16409d [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:39.852926 15516 raft_consensus.cc:2454] T f0f7177c76ec408ab20f70d30f6836cd P fc4578a90e094dfd8f119ccd8ee2227a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a03e7efd0f5540a1a44160559e469cc1 in term 1.
I20260812 16:37:39.852936 15268 raft_consensus.cc:2454] T f0f7177c76ec408ab20f70d30f6836cd P 87509834f1fc44a69dc033288c16409d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a03e7efd0f5540a1a44160559e469cc1 in term 1.
I20260812 16:37:39.853143 14911 leader_election.cc:302] T f0f7177c76ec408ab20f70d30f6836cd P a03e7efd0f5540a1a44160559e469cc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a03e7efd0f5540a1a44160559e469cc1, fc4578a90e094dfd8f119ccd8ee2227a; no voters:
I20260812 16:37:39.853277 15773 raft_consensus.cc:2781] T f0f7177c76ec408ab20f70d30f6836cd P a03e7efd0f5540a1a44160559e469cc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:37:39.853363 15773 raft_consensus.cc:686] T f0f7177c76ec408ab20f70d30f6836cd P a03e7efd0f5540a1a44160559e469cc1 [term 1 LEADER]: Becoming Leader. State: Replica: a03e7efd0f5540a1a44160559e469cc1, State: Running, Role: LEADER
I20260812 16:37:39.853420 15773 consensus_queue.cc:237] T f0f7177c76ec408ab20f70d30f6836cd P a03e7efd0f5540a1a44160559e469cc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } }
I20260812 16:37:39.854030 14751 catalog_manager.cc:5495] T f0f7177c76ec408ab20f70d30f6836cd P a03e7efd0f5540a1a44160559e469cc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a03e7efd0f5540a1a44160559e469cc1 (127.0.253.193). New cstate: current_term: 1 leader_uuid: "a03e7efd0f5540a1a44160559e469cc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03e7efd0f5540a1a44160559e469cc1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 } health_report { overall_health: UNKNOWN } } }
I20260812 16:37:39.862864 15651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:39.860825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34920 (local address 127.0.253.193:45315)
0812 16:37:39.860877 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:39.860879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:39.860886 (+ 7us) server_negotiation.cc:407] Connection header received
0812 16:37:39.860952 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:39.860952 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:39.860960 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:39.860999 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:37:39.861107 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.861558 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.862276 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.862452 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.862504 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:39.862645 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:37:39.862672 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:37:39.862711 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:39.862760 (+ 49us) server_negotiation.cc:299] Negotiation successful
0812 16:37:39.862778 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 16:37:39.866742 15651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:39.864930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:34897 (local address 127.0.253.193:45315)
0812 16:37:39.864985 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:39.864987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:39.864994 (+ 7us) server_negotiation.cc:407] Connection header received
0812 16:37:39.865027 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:39.865028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:39.865036 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:39.865087 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:37:39.865156 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.865569 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.866240 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.866404 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.866452 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:39.866563 (+ 111us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:37:39.866591 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:37:39.866632 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:39.866664 (+ 32us) server_negotiation.cc:299] Negotiation successful
0812 16:37:39.866683 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":30}
I20260812 16:37:39.866794 15800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:39.864877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:45315 (local address 127.0.253.193:34897)
0812 16:37:39.864927 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:39.864938 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:39.864988 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:39.865092 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:39.865092 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:39.865093 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 16:37:39.865124 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:39.865124 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.865655 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.865655 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:39.866172 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:39.866173 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:39.866467 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:39.866468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:39.866505 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:39.866510 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 16:37:39.866631 (+ 121us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 16:37:39.866633 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 16:37:39.866663 (+ 30us) client_negotiation.cc:240] Negotiation successful
0812 16:37:39.866704 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":2}
I20260812 16:37:39.868547 15516 raft_consensus.cc:1261] T f0f7177c76ec408ab20f70d30f6836cd P fc4578a90e094dfd8f119ccd8ee2227a [term 1 FOLLOWER]: Refusing update from remote peer a03e7efd0f5540a1a44160559e469cc1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:37:39.868595 15268 raft_consensus.cc:1261] T f0f7177c76ec408ab20f70d30f6836cd P 87509834f1fc44a69dc033288c16409d [term 1 FOLLOWER]: Refusing update from remote peer a03e7efd0f5540a1a44160559e469cc1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:37:39.868769 15773 consensus_queue.cc:1035] T f0f7177c76ec408ab20f70d30f6836cd P a03e7efd0f5540a1a44160559e469cc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc4578a90e094dfd8f119ccd8ee2227a" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:37:39.868878 15672 consensus_queue.cc:1035] T f0f7177c76ec408ab20f70d30f6836cd P a03e7efd0f5540a1a44160559e469cc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87509834f1fc44a69dc033288c16409d" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 33615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:37:39.881881 14751 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35274:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:37:39.881973 14751 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35274:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:37:39.882586 14751 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d520c8ddc0ff4a1e9d90dfab76998e47: Sending DeleteTablet for 3 replicas of tablet f0f7177c76ec408ab20f70d30f6836cd
I20260812 16:37:39.882853 15480 tablet_service.cc:1514] Processing DeleteTablet for tablet f0f7177c76ec408ab20f70d30f6836cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:39 UTC) from {username='oryx', principal='oryx/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:48312
I20260812 16:37:39.882900 15236 tablet_service.cc:1514] Processing DeleteTablet for tablet f0f7177c76ec408ab20f70d30f6836cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:39 UTC) from {username='oryx', principal='oryx/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:40278
I20260812 16:37:39.882982 15806 tablet_replica.cc:330] T f0f7177c76ec408ab20f70d30f6836cd P 87509834f1fc44a69dc033288c16409d: stopping tablet replica
I20260812 16:37:39.883047 15806 raft_consensus.cc:2229] T f0f7177c76ec408ab20f70d30f6836cd P 87509834f1fc44a69dc033288c16409d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:37:39.883108 15806 raft_consensus.cc:2258] T f0f7177c76ec408ab20f70d30f6836cd P 87509834f1fc44a69dc033288c16409d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:39.883062 14997 tablet_service.cc:1514] Processing DeleteTablet for tablet f0f7177c76ec408ab20f70d30f6836cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:37:39 UTC) from {username='oryx', principal='oryx/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:34898
I20260812 16:37:39.883183 15810 tablet_replica.cc:330] T f0f7177c76ec408ab20f70d30f6836cd P a03e7efd0f5540a1a44160559e469cc1: stopping tablet replica
I20260812 16:37:39.883193 15805 tablet_replica.cc:330] T f0f7177c76ec408ab20f70d30f6836cd P fc4578a90e094dfd8f119ccd8ee2227a: stopping tablet replica
I20260812 16:37:39.883239 15810 raft_consensus.cc:2229] T f0f7177c76ec408ab20f70d30f6836cd P a03e7efd0f5540a1a44160559e469cc1 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:37:39.883244 15805 raft_consensus.cc:2229] T f0f7177c76ec408ab20f70d30f6836cd P fc4578a90e094dfd8f119ccd8ee2227a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:37:39.883275 15805 raft_consensus.cc:2258] T f0f7177c76ec408ab20f70d30f6836cd P fc4578a90e094dfd8f119ccd8ee2227a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:39.883327 15810 raft_consensus.cc:2258] T f0f7177c76ec408ab20f70d30f6836cd P a03e7efd0f5540a1a44160559e469cc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:37:39.883641 15805 ts_tablet_manager.cc:1905] T f0f7177c76ec408ab20f70d30f6836cd P fc4578a90e094dfd8f119ccd8ee2227a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:37:39.884912 15810 ts_tablet_manager.cc:1905] T f0f7177c76ec408ab20f70d30f6836cd P a03e7efd0f5540a1a44160559e469cc1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:37:39.885159 15805 ts_tablet_manager.cc:1918] T f0f7177c76ec408ab20f70d30f6836cd P fc4578a90e094dfd8f119ccd8ee2227a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:37:39.885228 15805 log.cc:1196] T f0f7177c76ec408ab20f70d30f6836cd P fc4578a90e094dfd8f119ccd8ee2227a: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-2/wal/wals/f0f7177c76ec408ab20f70d30f6836cd
I20260812 16:37:39.885465 15805 ts_tablet_manager.cc:1939] T f0f7177c76ec408ab20f70d30f6836cd P fc4578a90e094dfd8f119ccd8ee2227a: Deleting consensus metadata
I20260812 16:37:39.885874 14718 catalog_manager.cc:4841] TS fc4578a90e094dfd8f119ccd8ee2227a (127.0.253.195:38573): tablet f0f7177c76ec408ab20f70d30f6836cd (table test-table [id=68887d8649ce4282a20cbb5165b8f5ac]) successfully deleted
I20260812 16:37:39.886340 15810 ts_tablet_manager.cc:1918] T f0f7177c76ec408ab20f70d30f6836cd P a03e7efd0f5540a1a44160559e469cc1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:37:39.886415 15810 log.cc:1196] T f0f7177c76ec408ab20f70d30f6836cd P a03e7efd0f5540a1a44160559e469cc1: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-0/wal/wals/f0f7177c76ec408ab20f70d30f6836cd
I20260812 16:37:39.886619 15810 ts_tablet_manager.cc:1939] T f0f7177c76ec408ab20f70d30f6836cd P a03e7efd0f5540a1a44160559e469cc1: Deleting consensus metadata
I20260812 16:37:39.886953 14721 catalog_manager.cc:4841] TS a03e7efd0f5540a1a44160559e469cc1 (127.0.253.193:45315): tablet f0f7177c76ec408ab20f70d30f6836cd (table test-table [id=68887d8649ce4282a20cbb5165b8f5ac]) successfully deleted
I20260812 16:37:39.887327 15806 ts_tablet_manager.cc:1905] T f0f7177c76ec408ab20f70d30f6836cd P 87509834f1fc44a69dc033288c16409d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:37:39.887442 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 14875
I20260812 16:37:39.888912 15806 ts_tablet_manager.cc:1918] T f0f7177c76ec408ab20f70d30f6836cd P 87509834f1fc44a69dc033288c16409d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:37:39.888959 15806 log.cc:1196] T f0f7177c76ec408ab20f70d30f6836cd P 87509834f1fc44a69dc033288c16409d: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552619305671-1015-0/minicluster-data/ts-1/wal/wals/f0f7177c76ec408ab20f70d30f6836cd
I20260812 16:37:39.889149 15806 ts_tablet_manager.cc:1939] T f0f7177c76ec408ab20f70d30f6836cd P 87509834f1fc44a69dc033288c16409d: Deleting consensus metadata
I20260812 16:37:39.900297 14721 catalog_manager.cc:4841] TS 87509834f1fc44a69dc033288c16409d (127.0.253.194:33615): tablet f0f7177c76ec408ab20f70d30f6836cd (table test-table [id=68887d8649ce4282a20cbb5165b8f5ac]) successfully deleted
I20260812 16:37:39.907158 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 15126
I20260812 16:37:39.926357 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 15370
I20260812 16:37:39.936997 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 14681
2026-08-12T16:37:39Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3354 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:37:40 dist-test-slave-w4v5 krb5kdc[15868](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:37:40 dist-test-slave-w4v5 krb5kdc[15868](info): set up 2 sockets
Aug 12 16:37:40 dist-test-slave-w4v5 krb5kdc[15868](info): commencing operation
krb5kdc: starting...
W20260812 16:37:41.250880 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.281s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:37:41 dist-test-slave-w4v5 krb5kdc[15868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:37:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:37:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
I20260812 16:37:41.378141 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:37319
--webserver_interface=127.0.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:37299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:37319
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:41.467875 15889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:41.468086 15889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:41.468116 15889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:41.469310 15889 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:37:41.469353 15889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:41.469368 15889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:41.469380 15889 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:37:41.469393 15889 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:37:41.470947 15889 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:37299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:37319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:37319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:41.471249 15889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:41.471505 15889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:41.476742 15902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:41.476755 15898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:41.476750 15904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:41.476838 15889 server_base.cc:1034] running on GCE node
I20260812 16:37:41.477249 15889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:41.477546 15889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:41.478736 15889 hybrid_clock.cc:648] HybridClock initialized: now 1786552661478706 us; error 48 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-12T16:37:41Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1537 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:37:41 dist-test-slave-w4v5 krb5kdc[15915](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:37:41 dist-test-slave-w4v5 krb5kdc[15915](info): set up 2 sockets
Aug 12 16:37:41 dist-test-slave-w4v5 krb5kdc[15915](info): commencing operation
krb5kdc: starting...
W20260812 16:37:42.811234 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.304s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:37:42 dist-test-slave-w4v5 krb5kdc[15915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552662, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:37:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:37:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
I20260812 16:37:42.943439 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:36149
--webserver_interface=127.0.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:35285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:36149
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:43.027225 15947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:43.027472 15947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:43.027509 15947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:43.028654 15947 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:37:43.028730 15947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:43.028755 15947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:43.028776 15947 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:37:43.028807 15947 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:37:43.030346 15947 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:35285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:36149
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:36149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:43.030669 15947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:43.030927 15947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:43.036477 15964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:43.036461 15953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:43.036492 15947 server_base.cc:1034] running on GCE node
W20260812 16:37:43.036461 15959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:43.037111 15947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:43.037428 15947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:43.038581 15947 hybrid_clock.cc:648] HybridClock initialized: now 1786552663038558 us; error 45 us; skew 500 ppm
Aug 12 16:37:43 dist-test-slave-w4v5 krb5kdc[15915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552663, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:43.045845 15947 init.cc:377] Logged in from keytab as kudu/127.0.253.254@KRBTEST.COM (short username kudu)
I20260812 16:37:43.046367 15947 webserver.cc:460] Webserver started at http://127.0.253.254:45037/ using document root <none> and password file <none>
I20260812 16:37:43.046634 15947 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:43.046684 15947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:43.046849 15947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:43.048008 15947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "96147f274d4b4fb990dda3b220e3356a"
format_stamp: "Formatted at 2026-08-12 16:37:43 on dist-test-slave-w4v5"
server_key: "402ebc1c683d3353f2db3f9790c547ab"
server_key_iv: "cf5d6cb6553942312bb39ac8a448194a"
server_key_version: "encryptionkey@0"
I20260812 16:37:43.048389 15947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "96147f274d4b4fb990dda3b220e3356a"
format_stamp: "Formatted at 2026-08-12 16:37:43 on dist-test-slave-w4v5"
server_key: "402ebc1c683d3353f2db3f9790c547ab"
server_key_iv: "cf5d6cb6553942312bb39ac8a448194a"
server_key_version: "encryptionkey@0"
I20260812 16:37:43.049799 15947 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:37:43.050654 15975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:43.050882 15947 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:43.050998 15947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "96147f274d4b4fb990dda3b220e3356a"
format_stamp: "Formatted at 2026-08-12 16:37:43 on dist-test-slave-w4v5"
server_key: "402ebc1c683d3353f2db3f9790c547ab"
server_key_iv: "cf5d6cb6553942312bb39ac8a448194a"
server_key_version: "encryptionkey@0"
I20260812 16:37:43.051115 15947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:43.063156 15947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:43.065555 15947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:43.065730 15947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:43.069932 15947 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:36149
I20260812 16:37:43.069967 16087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:36149 every 8 connection(s)
I20260812 16:37:43.070335 15947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:37:43.071391 16088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:43.073628 16088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a: Bootstrap starting.
I20260812 16:37:43.074180 16088 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:43.074429 16088 log.cc:824] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:43.075167 16088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a: No bootstrap required, opened a new log
I20260812 16:37:43.076752 16088 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96147f274d4b4fb990dda3b220e3356a" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36149 } }
I20260812 16:37:43.076893 16088 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:43.076916 16088 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96147f274d4b4fb990dda3b220e3356a, State: Initialized, Role: FOLLOWER
I20260812 16:37:43.077018 16088 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96147f274d4b4fb990dda3b220e3356a" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36149 } }
I20260812 16:37:43.077111 16088 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:37:43.077160 16088 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:37:43.077217 16088 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:43.077757 16088 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96147f274d4b4fb990dda3b220e3356a" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36149 } }
I20260812 16:37:43.077879 16088 leader_election.cc:302] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96147f274d4b4fb990dda3b220e3356a; no voters:
I20260812 16:37:43.078055 16088 leader_election.cc:288] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:37:43.078136 16095 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:37:43.078351 16088 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:37:43.078495 16095 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a [term 1 LEADER]: Becoming Leader. State: Replica: 96147f274d4b4fb990dda3b220e3356a, State: Running, Role: LEADER
I20260812 16:37:43.078605 16095 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96147f274d4b4fb990dda3b220e3356a" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36149 } }
I20260812 16:37:43.078874 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 15947
I20260812 16:37:43.079020 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:37:43.079023 16097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96147f274d4b4fb990dda3b220e3356a. Latest consensus state: current_term: 1 leader_uuid: "96147f274d4b4fb990dda3b220e3356a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96147f274d4b4fb990dda3b220e3356a" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36149 } } }
I20260812 16:37:43.079137 16097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:43.079149 16096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96147f274d4b4fb990dda3b220e3356a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96147f274d4b4fb990dda3b220e3356a" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36149 } } }
I20260812 16:37:43.079303 16096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:43.079309 1015 external_mini_cluster.cc:1428] Setting key 6a04963642171979d8f115bdbaef6d81
I20260812 16:37:43.079469 16112 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:37:43.080124 16112 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:37:43.084718 16112 catalog_manager.cc:1347] Generated new cluster ID: d4b34073cd7644ccbe8d25350bea6168
I20260812 16:37:43.084776 16112 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 16:37:43 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552662, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:43.089787 16127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:43.079822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46258 (local address 127.0.253.254:36149)
0812 16:37:43.079989 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:43.080000 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:43.080033 (+ 33us) server_negotiation.cc:407] Connection header received
0812 16:37:43.080165 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:43.080172 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:43.080525 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:43.081338 (+ 813us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:43.081413 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:43.082309 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:43.083111 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:43.083720 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:43.083861 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:43.086378 (+ 2517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:43.086383 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:43.086383 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:43.086395 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:43.088179 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:43.088565 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:43.088568 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:43.088569 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:43.088670 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:43.088884 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:43.088885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:43.088886 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:43.089136 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:43.089309 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:43.089405 (+ 96us) server_negotiation.cc:299] Negotiation successful
0812 16:37:43.089553 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":71,"threads_started":1}
I20260812 16:37:43.105245 16112 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:37:43.105773 16112 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:37:43.109169 16112 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a: Generated new TSK 0
I20260812 16:37:43.109357 16112 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab.
Entry for principal kudu/127.0.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab.
I20260812 16:37:43.145824 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.193:0
--local_ip_for_outbound_sockets=127.0.253.193
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:36149
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:35285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:43.229903 16138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:43.230103 16138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:43.230131 16138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:43.231338 16138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:43.231379 16138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:43.231405 16138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.193
I20260812 16:37:43.232903 16138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:35285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:36149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.193
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:43.233090 16138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:43.233260 16138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:43.238261 16144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:43.238338 16150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:43.238266 16147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:43.238744 16138 server_base.cc:1034] running on GCE node
I20260812 16:37:43.238943 16138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:43.239214 16138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:43.240365 16138 hybrid_clock.cc:648] HybridClock initialized: now 1786552663240351 us; error 29 us; skew 500 ppm
Aug 12 16:37:43 dist-test-slave-w4v5 krb5kdc[15915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552663, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:43.242915 16138 init.cc:377] Logged in from keytab as kudu/127.0.253.193@KRBTEST.COM (short username kudu)
I20260812 16:37:43.243405 16138 webserver.cc:460] Webserver started at http://127.0.253.193:43083/ using document root <none> and password file <none>
I20260812 16:37:43.243649 16138 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:43.243695 16138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:43.243893 16138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:43.244992 16138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-0/data/instance:
uuid: "8abe8cb8f6cd4bbc90aa4efc14ebb621"
format_stamp: "Formatted at 2026-08-12 16:37:43 on dist-test-slave-w4v5"
server_key: "def996ef60de6f7e40eea1c38fd9f742"
server_key_iv: "27783ecd0e0d166cd39105b00a85d31b"
server_key_version: "encryptionkey@0"
I20260812 16:37:43.245357 16138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance:
uuid: "8abe8cb8f6cd4bbc90aa4efc14ebb621"
format_stamp: "Formatted at 2026-08-12 16:37:43 on dist-test-slave-w4v5"
server_key: "def996ef60de6f7e40eea1c38fd9f742"
server_key_iv: "27783ecd0e0d166cd39105b00a85d31b"
server_key_version: "encryptionkey@0"
I20260812 16:37:43.246744 16138 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:37:43.247507 16174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:43.247686 16138 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:43.247787 16138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-0/wal
uuid: "8abe8cb8f6cd4bbc90aa4efc14ebb621"
format_stamp: "Formatted at 2026-08-12 16:37:43 on dist-test-slave-w4v5"
server_key: "def996ef60de6f7e40eea1c38fd9f742"
server_key_iv: "27783ecd0e0d166cd39105b00a85d31b"
server_key_version: "encryptionkey@0"
I20260812 16:37:43.247843 16138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:43.263720 16138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:43.265758 16138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:43.265867 16138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:43.266055 16138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:37:43.266398 16138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:43.266434 16138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:43.266479 16138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:43.266507 16138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:43.272399 16138 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.193:45651
I20260812 16:37:43.272487 16377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.193:45651 every 8 connection(s)
I20260812 16:37:43.272814 16138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
Aug 12 16:37:43 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552663, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:43.281647 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 16138
I20260812 16:37:43.281765 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance
I20260812 16:37:43.282004 1015 external_mini_cluster.cc:1428] Setting key f4d3bcc54af445546ac48be9a5f3dd68
I20260812 16:37:43.283149 16127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:43.273662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:44747 (local address 127.0.253.254:36149)
0812 16:37:43.273747 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:43.273750 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:43.273938 (+ 188us) server_negotiation.cc:407] Connection header received
0812 16:37:43.274412 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:43.274413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:43.274428 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:43.274485 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:43.275509 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:43.276018 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:43.277328 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:43.277614 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:43.277686 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:43.280078 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:43.280080 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:43.280081 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:43.280094 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:43.281445 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:43.281958 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:43.281960 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:43.281960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:43.282020 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:43.282360 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:43.282361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:43.282362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:43.282485 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:43.282614 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:43.282943 (+ 329us) server_negotiation.cc:299] Negotiation successful
0812 16:37:43.283037 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 16:37:43.283370 16382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:43.273610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.193:44747)
0812 16:37:43.273816 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:43.273853 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:43.274327 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:43.274545 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:43.274548 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:43.275230 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:43.275429 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:43.275434 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:43.276102 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:43.276102 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:43.277249 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:43.277252 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:43.277698 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:43.277698 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:43.277834 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:43.278378 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:43.278387 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:43.279970 (+ 1583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:43.281539 (+ 1569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:43.281547 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:43.281549 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:43.281875 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:43.282136 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:43.282137 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:43.282137 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:43.282277 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:43.282609 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:43.282617 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:43.282786 (+ 169us) client_negotiation.cc:769] Sending connection context
0812 16:37:43.282919 (+ 133us) client_negotiation.cc:240] Negotiation successful
0812 16:37:43.283117 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":126,"threads_started":1}
I20260812 16:37:43.283878 16378 heartbeater.cc:347] Connected to a master server at 127.0.253.254:36149
I20260812 16:37:43.283987 16378 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:43.284173 16378 heartbeater.cc:511] Master 127.0.253.254:36149 requested a full tablet report, sending...
I20260812 16:37:43.284765 16016 ts_manager.cc:194] Registered new tserver with Master: 8abe8cb8f6cd4bbc90aa4efc14ebb621 (127.0.253.193:45651)
I20260812 16:37:43.285384 16016 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.193@KRBTEST.COM'} at 127.0.253.193:44747
WARNING: no policy specified for kudu/127.0.253.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab.
Entry for principal kudu/127.0.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab.
I20260812 16:37:43.309496 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.194:0
--local_ip_for_outbound_sockets=127.0.253.194
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:36149
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:35285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:43.397382 16386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:43.397704 16386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:43.397742 16386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:43.399315 16386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:43.399376 16386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:43.399421 16386 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.194
I20260812 16:37:43.400884 16386 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:35285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:36149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.194
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:43.401175 16386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:43.401414 16386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:43.406582 16395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:43.406617 16386 server_base.cc:1034] running on GCE node
W20260812 16:37:43.406582 16400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:43.406669 16394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:43.407061 16386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:43.407377 16386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:43.408547 16386 hybrid_clock.cc:648] HybridClock initialized: now 1786552663408540 us; error 37 us; skew 500 ppm
Aug 12 16:37:43 dist-test-slave-w4v5 krb5kdc[15915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552663, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:43.411315 16386 init.cc:377] Logged in from keytab as kudu/127.0.253.194@KRBTEST.COM (short username kudu)
I20260812 16:37:43.411835 16386 webserver.cc:460] Webserver started at http://127.0.253.194:42731/ using document root <none> and password file <none>
I20260812 16:37:43.412092 16386 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:43.412171 16386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:43.412317 16386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:43.413362 16386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-1/data/instance:
uuid: "047bebacbf3a47629aa88fc1a60fd36f"
format_stamp: "Formatted at 2026-08-12 16:37:43 on dist-test-slave-w4v5"
server_key: "5b4188e520637aa28ad432fae9db502c"
server_key_iv: "0a4642e48c9feecea0bd0931efba9300"
server_key_version: "encryptionkey@0"
I20260812 16:37:43.413753 16386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance:
uuid: "047bebacbf3a47629aa88fc1a60fd36f"
format_stamp: "Formatted at 2026-08-12 16:37:43 on dist-test-slave-w4v5"
server_key: "5b4188e520637aa28ad432fae9db502c"
server_key_iv: "0a4642e48c9feecea0bd0931efba9300"
server_key_version: "encryptionkey@0"
I20260812 16:37:43.415146 16386 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 16:37:43.415891 16413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:43.416102 16386 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:37:43.416213 16386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-1/wal
uuid: "047bebacbf3a47629aa88fc1a60fd36f"
format_stamp: "Formatted at 2026-08-12 16:37:43 on dist-test-slave-w4v5"
server_key: "5b4188e520637aa28ad432fae9db502c"
server_key_iv: "0a4642e48c9feecea0bd0931efba9300"
server_key_version: "encryptionkey@0"
I20260812 16:37:43.416296 16386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:43.432699 16386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:43.434727 16386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:43.434892 16386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:43.435184 16386 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:37:43.435503 16386 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:43.435559 16386 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:43.435604 16386 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:43.435633 16386 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:43.441669 16386 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.194:41951
I20260812 16:37:43.441725 16617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.194:41951 every 8 connection(s)
I20260812 16:37:43.442071 16386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
I20260812 16:37:43.445046 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 16386
I20260812 16:37:43.445196 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance
I20260812 16:37:43.445382 1015 external_mini_cluster.cc:1428] Setting key 716ba2cf0a495088a0fe18d0c3f17a06
Aug 12 16:37:43 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552663, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:43.452054 16127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:43.442800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:37043 (local address 127.0.253.254:36149)
0812 16:37:43.442847 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:43.442849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:43.443100 (+ 251us) server_negotiation.cc:407] Connection header received
0812 16:37:43.443616 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:43.443618 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:43.443631 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:43.443687 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:43.444633 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:43.445135 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:43.446456 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:43.446760 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:43.446842 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:43.449053 (+ 2211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:43.449055 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:43.449055 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:43.449063 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:43.450605 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:43.451000 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:43.451001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:43.451002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:43.451066 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:43.451376 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:43.451377 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:43.451377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:43.451533 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:43.451600 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:43.451879 (+ 279us) server_negotiation.cc:299] Negotiation successful
0812 16:37:43.451965 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 16:37:43.452242 16624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:43.442764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.194:37043)
0812 16:37:43.442927 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:43.442961 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:43.443521 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:43.443757 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:43.443760 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:43.444336 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:43.444532 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:43.444537 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:43.445274 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:43.445275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:43.446367 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:43.446370 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:43.446873 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:43.446874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:43.447016 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:43.447603 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:43.447611 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:43.448951 (+ 1340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:43.450687 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:43.450694 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:43.450697 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:43.450921 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:43.451171 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:43.451172 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:43.451172 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:43.451298 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:43.451595 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:43.451602 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:43.451742 (+ 140us) client_negotiation.cc:769] Sending connection context
0812 16:37:43.451845 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 16:37:43.452000 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":97,"threads_started":1}
I20260812 16:37:43.452845 16620 heartbeater.cc:347] Connected to a master server at 127.0.253.254:36149
I20260812 16:37:43.452953 16620 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:43.453156 16620 heartbeater.cc:511] Master 127.0.253.254:36149 requested a full tablet report, sending...
I20260812 16:37:43.453580 16016 ts_manager.cc:194] Registered new tserver with Master: 047bebacbf3a47629aa88fc1a60fd36f (127.0.253.194:41951)
I20260812 16:37:43.453965 16016 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.194@KRBTEST.COM'} at 127.0.253.194:37043
WARNING: no policy specified for kudu/127.0.253.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab.
Entry for principal kudu/127.0.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab.
I20260812 16:37:43.473767 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.195:0
--local_ip_for_outbound_sockets=127.0.253.195
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:36149
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:35285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:43.558492 16629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:43.558724 16629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:43.558770 16629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:43.559926 16629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:43.559963 16629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:43.559989 16629 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.195
I20260812 16:37:43.561352 16629 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:35285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:36149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16629
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.195
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:43.561533 16629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:43.561705 16629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:43.566670 16637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:43.566679 16641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:43.566686 16636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:43.567149 16629 server_base.cc:1034] running on GCE node
I20260812 16:37:43.567376 16629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:43.567627 16629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:43.568789 16629 hybrid_clock.cc:648] HybridClock initialized: now 1786552663568776 us; error 30 us; skew 500 ppm
Aug 12 16:37:43 dist-test-slave-w4v5 krb5kdc[15915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552663, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:43.571530 16629 init.cc:377] Logged in from keytab as kudu/127.0.253.195@KRBTEST.COM (short username kudu)
I20260812 16:37:43.571957 16629 webserver.cc:460] Webserver started at http://127.0.253.195:37763/ using document root <none> and password file <none>
I20260812 16:37:43.572212 16629 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:43.572283 16629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:43.572429 16629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:43.573450 16629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-2/data/instance:
uuid: "96dcb39be96f49ccbb1af8fbb455bee6"
format_stamp: "Formatted at 2026-08-12 16:37:43 on dist-test-slave-w4v5"
server_key: "07b54f6c65bfa440c365d8ed421c4697"
server_key_iv: "e59e73be2fdb823916dc35a8f206609f"
server_key_version: "encryptionkey@0"
I20260812 16:37:43.573810 16629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance:
uuid: "96dcb39be96f49ccbb1af8fbb455bee6"
format_stamp: "Formatted at 2026-08-12 16:37:43 on dist-test-slave-w4v5"
server_key: "07b54f6c65bfa440c365d8ed421c4697"
server_key_iv: "e59e73be2fdb823916dc35a8f206609f"
server_key_version: "encryptionkey@0"
I20260812 16:37:43.575241 16629 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:37:43.575955 16653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:43.576126 16629 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:37:43.576227 16629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-2/wal
uuid: "96dcb39be96f49ccbb1af8fbb455bee6"
format_stamp: "Formatted at 2026-08-12 16:37:43 on dist-test-slave-w4v5"
server_key: "07b54f6c65bfa440c365d8ed421c4697"
server_key_iv: "e59e73be2fdb823916dc35a8f206609f"
server_key_version: "encryptionkey@0"
I20260812 16:37:43.576308 16629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:43.591919 16629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:43.594208 16629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:43.594368 16629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:43.594590 16629 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:37:43.594904 16629 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:43.594960 16629 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:43.595002 16629 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:43.595029 16629 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:43.600976 16629 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.195:40975
I20260812 16:37:43.601024 16850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.195:40975 every 8 connection(s)
I20260812 16:37:43.601370 16629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
Aug 12 16:37:43 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552663, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:37:43.609427 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 16629
I20260812 16:37:43.609540 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance
I20260812 16:37:43.609759 1015 external_mini_cluster.cc:1428] Setting key 2d9f65464f958e6ae94ff2c768366cbd
I20260812 16:37:43.610886 16127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:43.602077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:39919 (local address 127.0.253.254:36149)
0812 16:37:43.602126 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:43.602128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:43.602314 (+ 186us) server_negotiation.cc:407] Connection header received
0812 16:37:43.602799 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:43.602801 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:43.602811 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:43.602864 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:43.604014 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:43.604514 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:43.605467 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:43.605672 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:43.605730 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:43.607857 (+ 2127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:43.607859 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:43.607859 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:43.607866 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:43.609297 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:43.609801 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:43.609802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:43.609803 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:43.609890 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:43.610165 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:43.610166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:43.610167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:43.610286 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:43.610364 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:43.610709 (+ 345us) server_negotiation.cc:299] Negotiation successful
0812 16:37:43.610795 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 16:37:43.611024 16857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:43.602042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.195:39919)
0812 16:37:43.602200 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:43.602245 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:43.602710 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:43.602949 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:43.602954 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:43.603671 (+ 717us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:43.603912 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:43.603917 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:43.604597 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:43.604598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:43.605389 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:43.605391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:43.605754 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:43.605755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:43.605862 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:43.606308 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:43.606315 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:43.607776 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:43.609398 (+ 1622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:43.609405 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:43.609409 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:43.609721 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:43.609968 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:37:43.609969 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:37:43.609969 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:37:43.610089 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:37:43.610367 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:43.610375 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:43.610524 (+ 149us) client_negotiation.cc:769] Sending connection context
0812 16:37:43.610627 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 16:37:43.610800 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":93,"threads_started":1}
I20260812 16:37:43.611557 16853 heartbeater.cc:347] Connected to a master server at 127.0.253.254:36149
I20260812 16:37:43.611658 16853 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:43.611862 16853 heartbeater.cc:511] Master 127.0.253.254:36149 requested a full tablet report, sending...
I20260812 16:37:43.612293 16016 ts_manager.cc:194] Registered new tserver with Master: 96dcb39be96f49ccbb1af8fbb455bee6 (127.0.253.195:40975)
I20260812 16:37:43.612697 16016 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.195@KRBTEST.COM'} at 127.0.253.195:39919
I20260812 16:37:43.621416 1015 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.0.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.193.keytab.
Entry for principal oryx/127.0.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.193.keytab.
WARNING: no policy specified for oryx/127.0.253.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.253.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.194.keytab.
Entry for principal oryx/127.0.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.194.keytab.
WARNING: no policy specified for oryx/127.0.253.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.195.keytab.
Entry for principal oryx/127.0.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.195.keytab.
I20260812 16:37:43.697631 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 16138
I20260812 16:37:43.724098 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 16386
I20260812 16:37:43.736017 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 16629
I20260812 16:37:43.761196 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 15947
I20260812 16:37:43.771767 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:36149
--webserver_interface=127.0.253.254
--webserver_port=45037
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:35285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:36149
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:37:43.857548 16870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:43.857793 16870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:43.857833 16870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:43.858999 16870 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:37:43.859069 16870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:43.859109 16870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:43.859131 16870 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:37:43.859163 16870 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:37:43.860852 16870 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:35285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:36149
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:36149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=45037
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:43.861148 16870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:43.861402 16870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 16:37:43.866798 16870 server_base.cc:1034] running on GCE node
W20260812 16:37:43.866752 16879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:43.866770 16881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:43.866752 16886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:43.867276 16870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:43.867586 16870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:43.868808 16870 hybrid_clock.cc:648] HybridClock initialized: now 1786552663868788 us; error 34 us; skew 500 ppm
Aug 12 16:37:43 dist-test-slave-w4v5 krb5kdc[15915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552663, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:43.871596 16870 init.cc:377] Logged in from keytab as kudu/127.0.253.254@KRBTEST.COM (short username kudu)
I20260812 16:37:43.872093 16870 webserver.cc:460] Webserver started at http://127.0.253.254:45037/ using document root <none> and password file <none>
I20260812 16:37:43.872366 16870 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:43.872462 16870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:43.873862 16870 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:43.874569 16898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:43.874749 16870 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:43.874866 16870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "96147f274d4b4fb990dda3b220e3356a"
format_stamp: "Formatted at 2026-08-12 16:37:43 on dist-test-slave-w4v5"
server_key: "402ebc1c683d3353f2db3f9790c547ab"
server_key_iv: "cf5d6cb6553942312bb39ac8a448194a"
server_key_version: "encryptionkey@0"
I20260812 16:37:43.875298 16870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:43.890964 16870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:43.893117 16870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:43.893297 16870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:43.897606 16870 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:36149
I20260812 16:37:43.897640 16998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:36149 every 8 connection(s)
I20260812 16:37:43.898036 16870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:37:43.899669 17002 sys_catalog.cc:263] Verifying existing consensus state
I20260812 16:37:43.900278 17002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a: Bootstrap starting.
I20260812 16:37:43.907698 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 16870
I20260812 16:37:43.908063 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.193:45651
--local_ip_for_outbound_sockets=127.0.253.193
--tserver_master_addrs=127.0.253.254:36149
--webserver_port=43083
--webserver_interface=127.0.253.193
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:35285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.0.253.193
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:37:43.909111 17002 log.cc:824] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:43.911057 17002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 16:37:43.911386 17002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a: Bootstrap complete.
I20260812 16:37:43.915418 17002 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96147f274d4b4fb990dda3b220e3356a" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36149 } }
I20260812 16:37:43.915823 17002 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96147f274d4b4fb990dda3b220e3356a, State: Initialized, Role: FOLLOWER
I20260812 16:37:43.916056 17002 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96147f274d4b4fb990dda3b220e3356a" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36149 } }
I20260812 16:37:43.916178 17002 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:37:43.916255 17002 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:37:43.916330 17002 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a [term 1 FOLLOWER]: Advancing to term 2
I20260812 16:37:43.917971 17002 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96147f274d4b4fb990dda3b220e3356a" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36149 } }
I20260812 16:37:43.918103 17002 leader_election.cc:302] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96147f274d4b4fb990dda3b220e3356a; no voters:
I20260812 16:37:43.918313 17002 leader_election.cc:288] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 16:37:43.918398 17015 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a [term 2 FOLLOWER]: Leader election won for term 2
I20260812 16:37:43.918581 17015 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a [term 2 LEADER]: Becoming Leader. State: Replica: 96147f274d4b4fb990dda3b220e3356a, State: Running, Role: LEADER
I20260812 16:37:43.918627 17002 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:37:43.918773 17015 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96147f274d4b4fb990dda3b220e3356a" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36149 } }
I20260812 16:37:43.919153 17017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96147f274d4b4fb990dda3b220e3356a. Latest consensus state: current_term: 2 leader_uuid: "96147f274d4b4fb990dda3b220e3356a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96147f274d4b4fb990dda3b220e3356a" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36149 } } }
I20260812 16:37:43.919154 17016 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "96147f274d4b4fb990dda3b220e3356a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96147f274d4b4fb990dda3b220e3356a" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36149 } } }
I20260812 16:37:43.919258 17017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:43.919265 17016 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:43.919718 17028 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:37:43.920229 17028 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:37:43.920542 17028 catalog_manager.cc:1259] Loaded cluster ID: d4b34073cd7644ccbe8d25350bea6168
I20260812 16:37:43.920589 17028 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:37:43.921751 17028 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:37:43.921945 17028 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 96147f274d4b4fb990dda3b220e3356a: Loaded TSK: 0
I20260812 16:37:43.922175 17028 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 16:37:43.994004 17010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:43.994287 17010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:43.994324 17010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:43.995522 17010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:43.995591 17010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:43.995636 17010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.193
I20260812 16:37:43.997099 17010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:35285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.193.keytab
--principal=oryx/127.0.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.193:45651
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.253.193
--webserver_port=43083
--tserver_master_addrs=127.0.253.254:36149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17010
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.193
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:43.997357 17010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:43.997592 17010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:44.003692 17052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:44.003701 17055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:44.003809 17061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:44.004252 17010 server_base.cc:1034] running on GCE node
I20260812 16:37:44.004464 17010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:44.004740 17010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:44.005929 17010 hybrid_clock.cc:648] HybridClock initialized: now 1786552664005914 us; error 28 us; skew 500 ppm
Aug 12 16:37:44 dist-test-slave-w4v5 krb5kdc[15915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552664, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:44.009402 17010 init.cc:377] Logged in from keytab as oryx/127.0.253.193@KRBTEST.COM (short username oryx)
I20260812 16:37:44.009961 17010 webserver.cc:460] Webserver started at http://127.0.253.193:43083/ using document root <none> and password file <none>
I20260812 16:37:44.010273 17010 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:44.010356 17010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:44.012257 17010 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:37:44.013023 17072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:44.013271 17010 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:37:44.013396 17010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-0/wal
uuid: "8abe8cb8f6cd4bbc90aa4efc14ebb621"
format_stamp: "Formatted at 2026-08-12 16:37:43 on dist-test-slave-w4v5"
server_key: "def996ef60de6f7e40eea1c38fd9f742"
server_key_iv: "27783ecd0e0d166cd39105b00a85d31b"
server_key_version: "encryptionkey@0"
I20260812 16:37:44.013852 17010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:44.037500 17010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:44.040015 17010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:44.040180 17010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:44.040505 17010 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:37:44.040927 17010 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:44.040967 17010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:44.040992 17010 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:44.041038 17010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:44.048632 17010 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.193:45651
I20260812 16:37:44.048723 17263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.193:45651 every 8 connection(s)
I20260812 16:37:44.049081 17010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
I20260812 16:37:44.055172 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 17010
I20260812 16:37:44.055516 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.194:41951
--local_ip_for_outbound_sockets=127.0.253.194
--tserver_master_addrs=127.0.253.254:36149
--webserver_port=42731
--webserver_interface=127.0.253.194
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:35285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.0.253.194
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 16:37:44 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.193@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:44 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.193@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:44.059151 17269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:44.049899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.193:44661)
0812 16:37:44.050102 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:44.050153 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:44.050647 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:44.051185 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:44.051189 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:44.051792 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:44.052031 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:44.052036 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:44.053615 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:44.053616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:44.054414 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:44.054416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:44.055416 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:44.055417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:44.055569 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:44.056212 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:44.056221 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:44.058857 (+ 2636us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":91,"threads_started":1}
W20260812 16:37:44.059226 17270 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:44.049971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:44661 (local address 127.0.253.254:36149)
0812 16:37:44.050212 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:44.050222 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:44.050265 (+ 43us) server_negotiation.cc:407] Connection header received
0812 16:37:44.050860 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:44.050868 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:44.050928 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:44.051104 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:44.052152 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:44.053506 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:44.054508 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:44.055275 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:44.055401 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:44.058991 (+ 3590us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.193:44661: BlockingRecv error: recv got EOF from 127.0.253.193:44661 (error 108)
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":109,"threads_started":1}
W20260812 16:37:44.059516 17266 heartbeater.cc:644] Failed to heartbeat to 127.0.253.254:36149 (0 consecutive failures): Not authorized: Failed to ping master at 127.0.253.254:36149: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Aug 12 16:37:44 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.193@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:44 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.193@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:44.064299 17269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:44.059797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.193:33131)
0812 16:37:44.059850 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:44.059861 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:44.059906 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:44.060047 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:44.060048 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:44.060184 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:44.060228 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:44.060229 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:44.060867 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:44.060868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:44.061368 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:44.061369 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:44.061734 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:44.061735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:44.061800 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:44.062248 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:44.062253 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:44.064196 (+ 1943us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260812 16:37:44.064324 17270 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:44.059861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:33131 (local address 127.0.253.254:36149)
0812 16:37:44.059924 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:44.059927 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:44.059936 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:37:44.059957 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:44.059958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:44.059970 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:44.060044 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:44.060315 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:44.060787 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:44.061470 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:44.061662 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:44.061736 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:44.064284 (+ 2548us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.193:33131: BlockingRecv error: recv got EOF from 127.0.253.193:33131 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 12 16:37:44 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.193@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:44 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.193@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:44.069057 17269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:44.064689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.193:41245)
0812 16:37:44.064745 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:44.064755 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:44.064807 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:44.064965 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:44.064966 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:44.065102 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:44.065127 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:44.065128 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:44.065730 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:44.065731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:44.066227 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:44.066228 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:44.066606 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:44.066607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:44.066660 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:44.067184 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:44.067189 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:44.068966 (+ 1777us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":38}
W20260812 16:37:44.069118 17270 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:44.064750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:41245 (local address 127.0.253.254:36149)
0812 16:37:44.064794 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:44.064796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:44.064806 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:37:44.064875 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:44.064876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:44.064893 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:44.064961 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:44.065202 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:44.065648 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:44.066310 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:44.066489 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:44.066602 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:44.069052 (+ 2450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.193:41245: BlockingRecv error: recv got EOF from 127.0.253.193:41245 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260812 16:37:44.069425 17266 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 16:37:44.141289 17272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:44.141496 17272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:44.141556 17272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:44.142700 17272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:44.142750 17272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:44.142776 17272 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.194
I20260812 16:37:44.144179 17272 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:35285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.194.keytab
--principal=oryx/127.0.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.194:41951
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.253.194
--webserver_port=42731
--tserver_master_addrs=127.0.253.254:36149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.194
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:44.144371 17272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:44.144537 17272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:44.149686 17281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:44.149806 17279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:44.149757 17286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:44.149950 17272 server_base.cc:1034] running on GCE node
I20260812 16:37:44.150173 17272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:44.150419 17272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:44.151558 17272 hybrid_clock.cc:648] HybridClock initialized: now 1786552664151537 us; error 38 us; skew 500 ppm
Aug 12 16:37:44 dist-test-slave-w4v5 krb5kdc[15915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552664, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:44.154264 17272 init.cc:377] Logged in from keytab as oryx/127.0.253.194@KRBTEST.COM (short username oryx)
I20260812 16:37:44.154757 17272 webserver.cc:460] Webserver started at http://127.0.253.194:42731/ using document root <none> and password file <none>
I20260812 16:37:44.154997 17272 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:44.155056 17272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:44.156476 17272 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:44.157116 17298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:44.157330 17272 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:44.157425 17272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-1/wal
uuid: "047bebacbf3a47629aa88fc1a60fd36f"
format_stamp: "Formatted at 2026-08-12 16:37:43 on dist-test-slave-w4v5"
server_key: "5b4188e520637aa28ad432fae9db502c"
server_key_iv: "0a4642e48c9feecea0bd0931efba9300"
server_key_version: "encryptionkey@0"
I20260812 16:37:44.157763 17272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:44.168759 17272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:44.171066 17272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:44.171322 17272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:44.171571 17272 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:37:44.171888 17272 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:44.171923 17272 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:44.171945 17272 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:44.171958 17272 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:44.178045 17272 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.194:41951
I20260812 16:37:44.178113 17495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.194:41951 every 8 connection(s)
I20260812 16:37:44.178516 17272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
I20260812 16:37:44.182301 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 17272
I20260812 16:37:44.182633 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.195:40975
--local_ip_for_outbound_sockets=127.0.253.195
--tserver_master_addrs=127.0.253.254:36149
--webserver_port=37763
--webserver_interface=127.0.253.195
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:35285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.0.253.195
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 16:37:44 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.194@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:44 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.194@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:44.186096 17498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:44.179278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.194:42885)
0812 16:37:44.179474 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:44.179511 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:44.179979 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:44.180221 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:44.180223 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:44.180788 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:44.180985 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:44.180990 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:44.181712 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:44.181713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:44.182606 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:44.182609 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:44.183139 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:44.183141 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:44.183265 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:44.183800 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:44.183809 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:44.185958 (+ 2149us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":112,"threads_started":1}
W20260812 16:37:44.186245 17270 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:44.179303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:42885 (local address 127.0.253.254:36149)
0812 16:37:44.179395 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:44.179397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:44.179603 (+ 206us) server_negotiation.cc:407] Connection header received
0812 16:37:44.180091 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:44.180093 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:44.180108 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:44.180163 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:44.181106 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:44.181625 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:44.182702 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:44.183019 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:44.183128 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:44.186159 (+ 3031us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.194:42885: BlockingRecv error: recv got EOF from 127.0.253.194:42885 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260812 16:37:44.186394 17496 heartbeater.cc:644] Failed to heartbeat to 127.0.253.254:36149 (0 consecutive failures): Not authorized: Failed to ping master at 127.0.253.254:36149: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Aug 12 16:37:44 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.194@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:44 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.194@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:44.191300 17498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:44.186630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.194:34161)
0812 16:37:44.186681 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:44.186692 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:44.186742 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:44.186882 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:44.186883 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:44.187014 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:44.187055 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:44.187057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:44.188033 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:44.188034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:44.188513 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:44.188514 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:44.188997 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:44.188998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:44.189042 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:44.189470 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:44.189475 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:44.191257 (+ 1782us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 16:37:44.191455 17270 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:44.186692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:34161 (local address 127.0.253.254:36149)
0812 16:37:44.186747 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:44.186749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:44.186774 (+ 25us) server_negotiation.cc:407] Connection header received
0812 16:37:44.186798 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:44.186799 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:44.186809 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:44.186878 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:44.187162 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:44.187949 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:44.188594 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:44.188918 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:44.188999 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:44.191375 (+ 2376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.194:34161: BlockingRecv error: recv got EOF from 127.0.253.194:34161 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 12 16:37:44 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.194@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:44 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.194@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:44.195861 17498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:44.191538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.194:51451)
0812 16:37:44.191682 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:44.191691 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:44.191723 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:44.191926 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:44.191927 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:44.192049 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:44.192076 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:44.192077 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:44.192699 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:44.192700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:44.193188 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:44.193190 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:44.193564 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:44.193565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:44.193619 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:44.194008 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:44.194013 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:44.195785 (+ 1772us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":90,"threads_started":1}
W20260812 16:37:44.195940 17501 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:44.191644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:51451 (local address 127.0.253.254:36149)
0812 16:37:44.191799 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:44.191801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:44.191809 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:37:44.191835 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:44.191836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:44.191844 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:44.191911 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:44.192151 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:44.192617 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:44.193282 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:44.193480 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:44.193552 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:44.195867 (+ 2315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.194:51451: BlockingRecv error: recv got EOF from 127.0.253.194:51451 (error 108)
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":84,"threads_started":1}
W20260812 16:37:44.195978 17496 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 16:37:44.268468 17499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:44.268694 17499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:44.268738 17499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:44.269917 17499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:44.269982 17499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:44.270035 17499 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.195
I20260812 16:37:44.271605 17499 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:35285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/krb5kdc/oryx_127.0.253.195.keytab
--principal=oryx/127.0.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.195:40975
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.253.195
--webserver_port=37763
--tserver_master_addrs=127.0.253.254:36149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.195
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:44.271865 17499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:44.272105 17499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:44.277401 17515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:44.277441 17513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:44.277683 17499 server_base.cc:1034] running on GCE node
W20260812 16:37:44.277410 17510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:44.278059 17499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:44.278342 17499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:44.279521 17499 hybrid_clock.cc:648] HybridClock initialized: now 1786552664279509 us; error 45 us; skew 500 ppm
Aug 12 16:37:44 dist-test-slave-w4v5 krb5kdc[15915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552664, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:37:44.282061 17499 init.cc:377] Logged in from keytab as oryx/127.0.253.195@KRBTEST.COM (short username oryx)
I20260812 16:37:44.282549 17499 webserver.cc:460] Webserver started at http://127.0.253.195:37763/ using document root <none> and password file <none>
I20260812 16:37:44.282799 17499 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:44.282893 17499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:44.284333 17499 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:44.285012 17534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:44.285223 17499 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:44.285334 17499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-2/wal
uuid: "96dcb39be96f49ccbb1af8fbb455bee6"
format_stamp: "Formatted at 2026-08-12 16:37:43 on dist-test-slave-w4v5"
server_key: "07b54f6c65bfa440c365d8ed421c4697"
server_key_iv: "e59e73be2fdb823916dc35a8f206609f"
server_key_version: "encryptionkey@0"
I20260812 16:37:44.285708 17499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:44.294186 17499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:44.296402 17499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:44.296562 17499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:44.296838 17499 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:37:44.297154 17499 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:44.297214 17499 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:44.297253 17499 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:44.297282 17499 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:44.303344 17499 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.195:40975
I20260812 16:37:44.303401 17726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.195:40975 every 8 connection(s)
I20260812 16:37:44.303807 17499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
I20260812 16:37:44.308861 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 17499
Aug 12 16:37:44 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.195@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:44 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.195@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:44.311450 17731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:44.304575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.195:36773)
0812 16:37:44.304752 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:44.304787 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:44.305261 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:44.305484 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:44.305487 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:44.306048 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:44.306277 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:44.306282 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:44.306934 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:44.306935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:44.307813 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:44.307815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:44.308148 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:44.308149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:44.308275 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:44.308805 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:44.308815 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:44.311313 (+ 2498us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":110,"threads_started":1}
W20260812 16:37:44.311477 17501 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:44.304592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:36773 (local address 127.0.253.254:36149)
0812 16:37:44.304649 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:44.304651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:44.304883 (+ 232us) server_negotiation.cc:407] Connection header received
0812 16:37:44.305362 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:44.305363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:44.305377 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:44.305432 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:44.306363 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:44.306866 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:44.307890 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:44.308077 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:44.308131 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:44.311415 (+ 3284us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.195:36773: BlockingRecv error: recv got EOF from 127.0.253.195:36773 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260812 16:37:44.311753 17729 heartbeater.cc:644] Failed to heartbeat to 127.0.253.254:36149 (0 consecutive failures): Not authorized: Failed to ping master at 127.0.253.254:36149: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Aug 12 16:37:44 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.195@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:44 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.195@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:44.316295 17731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:44.311982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.195:33873)
0812 16:37:44.312027 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:44.312039 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:44.312090 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:44.312234 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:44.312234 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:44.312368 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:44.312410 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:44.312411 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:44.313000 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:44.313001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:44.313497 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:44.313499 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:44.313877 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:44.313878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:44.313958 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:44.314245 (+ 287us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:44.314250 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:44.316214 (+ 1964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260812 16:37:44.316373 17501 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:44.312003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:33873 (local address 127.0.253.254:36149)
0812 16:37:44.312040 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:44.312042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:44.312078 (+ 36us) server_negotiation.cc:407] Connection header received
0812 16:37:44.312146 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:44.312147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:44.312157 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:44.312229 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:44.312487 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:44.312920 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:44.313585 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:44.313802 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:44.313868 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:44.316308 (+ 2440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.195:33873: BlockingRecv error: recv got EOF from 127.0.253.195:33873 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 16:37:44.317299 17270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:44.309392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46272 (local address 127.0.253.254:36149)
0812 16:37:44.309460 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:44.309462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:44.309474 (+ 12us) server_negotiation.cc:407] Connection header received
0812 16:37:44.309552 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:44.309553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:44.309583 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:44.309653 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:44.310423 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:44.310941 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:44.312000 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:44.312197 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:44.312280 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:44.313502 (+ 1222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:44.313506 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:44.313507 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:37:44.313519 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:44.315473 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:44.315915 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:44.315918 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:44.315919 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:44.316044 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:37:44.316272 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:37:44.316273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:37:44.316273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:37:44.316819 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:44.316981 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:44.317058 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 16:37:44.317165 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
Aug 12 16:37:44 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.195@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:44 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.195@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:44.320942 17731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:44.316597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.195:39593)
0812 16:37:44.316645 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:44.316656 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:44.316693 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:44.316834 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:44.316835 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:44.316972 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:44.317005 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:44.317007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:44.317801 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:44.317802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:44.318452 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:44.318454 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:44.318913 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:44.318914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:44.318967 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:44.319338 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:44.319341 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:44.320903 (+ 1562us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 16:37:44.321072 17501 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:44.316654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:39593 (local address 127.0.253.254:36149)
0812 16:37:44.316687 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:44.316689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:44.316700 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:37:44.316776 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:44.316777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:44.316785 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:44.316828 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:44.317084 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:44.317719 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:44.318533 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:44.318829 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:44.318901 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:44.321005 (+ 2104us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.195:39593: BlockingRecv error: recv got EOF from 127.0.253.195:39593 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260812 16:37:44.321076 17729 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 16:37:45 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.193@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:45 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.193@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:45.075279 17740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:45.069852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.193:38821)
0812 16:37:45.070063 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:45.070075 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:45.070163 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:45.070378 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:45.070379 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:45.070691 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:45.070757 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:45.070760 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:45.071590 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:45.071591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:45.072155 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:45.072156 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:45.072535 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:45.072536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:45.072591 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:45.073126 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:45.073132 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:45.075160 (+ 2028us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":110,"threads_started":1}
W20260812 16:37:45.075317 17741 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:45.069988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:38821 (local address 127.0.253.254:36149)
0812 16:37:45.070164 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:45.070166 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:45.070179 (+ 13us) server_negotiation.cc:407] Connection header received
0812 16:37:45.070263 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:45.070264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:45.070278 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:45.070355 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:45.070835 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:45.071454 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:45.072237 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:45.072436 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:45.072495 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:45.075241 (+ 2746us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.193:38821: BlockingRecv error: recv got EOF from 127.0.253.193:38821 (error 108)
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":99,"threads_started":1}
Aug 12 16:37:45 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.194@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:45 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.194@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:45.201730 17742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:45.196387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.194:58389)
0812 16:37:45.196577 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:45.196592 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:45.196670 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:45.196897 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:45.196897 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:45.197093 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:45.197143 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:45.197145 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:45.197845 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:45.197847 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:45.198412 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:45.198413 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:45.198796 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:45.198797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:45.198858 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:45.199361 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:45.199366 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:45.201592 (+ 2226us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":94,"threads_started":1}
W20260812 16:37:45.201786 17741 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:45.196490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:58389 (local address 127.0.253.254:36149)
0812 16:37:45.196583 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:45.196585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:45.196648 (+ 63us) server_negotiation.cc:407] Connection header received
0812 16:37:45.196792 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:45.196793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:45.196809 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:45.196871 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:45.197244 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:45.197757 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:45.198503 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:45.198719 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:45.198796 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:45.201718 (+ 2922us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.194:58389: BlockingRecv error: recv got EOF from 127.0.253.194:58389 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 12 16:37:45 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.195@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:45 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.195@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:45.326956 17743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:45.321543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.195:40025)
0812 16:37:45.321695 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:45.321708 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:45.321781 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:45.321935 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:45.321935 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:45.322197 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:45.322263 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:45.322266 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:45.323192 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:45.323193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:45.323835 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:45.323838 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:45.324229 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:45.324230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:45.324292 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:45.324753 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:45.324758 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:45.326839 (+ 2081us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":94,"threads_started":1}
W20260812 16:37:45.326995 17741 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:45.321625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:40025 (local address 127.0.253.254:36149)
0812 16:37:45.321713 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:45.321715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:45.321781 (+ 66us) server_negotiation.cc:407] Connection header received
0812 16:37:45.321834 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:45.321835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:45.321854 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:45.321914 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:45.322376 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:45.323032 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:45.323930 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:45.324134 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:45.324194 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:45.326928 (+ 2734us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.195:40025: BlockingRecv error: recv got EOF from 127.0.253.195:40025 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 12 16:37:46 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.193@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:46 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.193@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:46.080943 17745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:46.075809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.193:35405)
0812 16:37:46.075985 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:46.076005 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:46.076064 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:46.076208 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:46.076209 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:46.076385 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:46.076438 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:46.076439 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:46.077147 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:46.077149 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:46.077734 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:46.077737 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:46.078187 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:46.078188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:46.078235 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:46.078704 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:46.078709 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:46.080882 (+ 2173us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":97,"threads_started":1}
W20260812 16:37:46.081115 17746 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:46.075922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:35405 (local address 127.0.253.254:36149)
0812 16:37:46.076055 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:46.076057 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:46.076067 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:37:46.076115 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:46.076115 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:46.076125 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:46.076202 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:46.076503 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:46.077052 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:46.077868 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:46.078112 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:46.078183 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:46.081002 (+ 2819us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.193:35405: BlockingRecv error: recv got EOF from 127.0.253.193:35405 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":82,"threads_started":1}
Aug 12 16:37:46 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.194@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:46 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.194@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:46.206971 17748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:46.202213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.194:56427)
0812 16:37:46.202368 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:46.202379 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:46.202437 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:46.202568 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:46.202568 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:46.202742 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:46.202792 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:46.202793 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:46.203512 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:46.203512 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:46.204063 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:46.204064 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:46.204392 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:46.204392 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:46.204434 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:46.204938 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:46.204943 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:46.206912 (+ 1969us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":101,"threads_started":1}
W20260812 16:37:46.207134 17746 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:46.202266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:56427 (local address 127.0.253.254:36149)
0812 16:37:46.202336 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:46.202339 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:46.202447 (+ 108us) server_negotiation.cc:407] Connection header received
0812 16:37:46.202476 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:46.202477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:46.202489 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:46.202539 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:46.202857 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:46.203442 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:46.204135 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:46.204324 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:46.204378 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:46.207016 (+ 2638us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.194:56427: BlockingRecv error: recv got EOF from 127.0.253.194:56427 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 12 16:37:46 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.195@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:46 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.195@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:46.332476 17750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:46.327522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.195:54407)
0812 16:37:46.327706 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:46.327720 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:46.327770 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:46.327910 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:46.327910 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:46.328099 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:46.328150 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:46.328152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:46.328865 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:46.328867 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:46.329407 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:46.329409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:46.329927 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:46.329929 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:46.329984 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:46.330427 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:46.330431 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:46.332418 (+ 1987us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":116,"threads_started":1}
W20260812 16:37:46.332818 17746 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:46.327704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:54407 (local address 127.0.253.254:36149)
0812 16:37:46.327775 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:46.327777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:46.327787 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:37:46.327822 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:46.327822 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:46.327833 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:46.327889 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:46.328229 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:46.328768 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:46.329516 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:46.329771 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:46.329847 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:46.332552 (+ 2705us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.195:54407: BlockingRecv error: recv got EOF from 127.0.253.195:54407 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 12 16:37:47 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.193@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:47 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.193@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:47.086529 17753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:47.081462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.193:60671)
0812 16:37:47.081659 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:47.081671 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:47.081731 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:47.081903 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:47.081904 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:47.082105 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:47.082171 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:47.082173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:47.082948 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:47.082949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:47.083567 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:47.083570 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:47.083999 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:47.084001 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:47.084066 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:47.084597 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:47.084602 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:47.086464 (+ 1862us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":131,"threads_started":1}
W20260812 16:37:47.086720 17754 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:47.081524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:60671 (local address 127.0.253.254:36149)
0812 16:37:47.081761 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:47.081763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:47.081773 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:37:47.081805 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:47.081806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:47.081818 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:47.081887 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:47.082266 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:47.082855 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:47.083672 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:47.083900 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:47.083971 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:47.086633 (+ 2662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.193:60671: BlockingRecv error: recv got EOF from 127.0.253.193:60671 (error 108)
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":182,"threads_started":1}
Aug 12 16:37:47 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.194@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:47 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.194@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:47.212282 17755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:47.207557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.194:58739)
0812 16:37:47.207734 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:47.207745 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:47.207807 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:47.207971 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:47.207971 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:47.208135 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:47.208186 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:47.208188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:47.208821 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:47.208821 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:47.209346 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:47.209347 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:47.209672 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:47.209673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:47.209717 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:47.210174 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:47.210179 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:47.212221 (+ 2042us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":118,"threads_started":1}
W20260812 16:37:47.212352 17754 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:47.207626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:58739 (local address 127.0.253.254:36149)
0812 16:37:47.207710 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:47.207713 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:47.207808 (+ 95us) server_negotiation.cc:407] Connection header received
0812 16:37:47.207877 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:47.207878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:47.207892 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:47.207961 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:47.208252 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:47.208751 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:47.209420 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:47.209603 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:47.209655 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:47.212303 (+ 2648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.194:58739: BlockingRecv error: recv got EOF from 127.0.253.194:58739 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 12 16:37:47 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.195@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:47 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.195@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:47.337785 17758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:47.333091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.195:47829)
0812 16:37:47.333279 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:47.333290 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:47.333354 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:47.333480 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:47.333481 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:47.333657 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:47.333707 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:47.333709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:47.334371 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:47.334372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:47.334895 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:47.334897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:47.335257 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:47.335258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:47.335302 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:47.335847 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:47.335853 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:47.337718 (+ 1865us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":126,"threads_started":1}
W20260812 16:37:47.337967 17754 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:47.333144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:47829 (local address 127.0.253.254:36149)
0812 16:37:47.333220 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:47.333223 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:47.333359 (+ 136us) server_negotiation.cc:407] Connection header received
0812 16:37:47.333404 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:47.333405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:47.333418 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:47.333459 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:47.333804 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:47.334303 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:47.334971 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:47.335185 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:47.335238 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:47.337852 (+ 2614us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.195:47829: BlockingRecv error: recv got EOF from 127.0.253.195:47829 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 12 16:37:48 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.193@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:48 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.193@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:48.097131 17759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:48.087131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.193:60515)
0812 16:37:48.087265 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:48.087276 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:48.087338 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:48.087468 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:48.087469 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:48.087650 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:48.087702 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:48.087704 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:48.088433 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:48.088434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:48.089179 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:48.089182 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:48.089666 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:48.089667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:48.089725 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:48.095108 (+ 5383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:48.095113 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:48.097029 (+ 1916us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":78,"threads_started":1}
W20260812 16:37:48.097200 17760 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:48.087216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:60515 (local address 127.0.253.254:36149)
0812 16:37:48.087352 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:48.087354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:48.087379 (+ 25us) server_negotiation.cc:407] Connection header received
0812 16:37:48.087402 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:48.087403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:48.087413 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:48.087465 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:48.087813 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:48.088326 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:48.089282 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:48.089567 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:48.089667 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:48.097128 (+ 7461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.193:60515: BlockingRecv error: recv got EOF from 127.0.253.193:60515 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":77,"threads_started":1}
Aug 12 16:37:48 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.194@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:48 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.194@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:48.218025 17761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:48.212911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.194:51733)
0812 16:37:48.213099 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:48.213110 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:48.213177 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:48.213339 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:48.213340 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:48.213594 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:48.213646 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:48.213648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:48.214375 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:48.214376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:48.214950 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:48.214952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:48.215449 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:48.215450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:48.215498 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:48.216086 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:48.216091 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:48.217963 (+ 1872us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":126,"threads_started":1}
W20260812 16:37:48.218240 17760 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:48.212971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:51733 (local address 127.0.253.254:36149)
0812 16:37:48.213045 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:48.213047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:48.213176 (+ 129us) server_negotiation.cc:407] Connection header received
0812 16:37:48.213228 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:48.213229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:48.213239 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:48.213294 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:48.213730 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:48.214272 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:48.215050 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:48.215369 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:48.215438 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:48.218169 (+ 2731us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.194:51733: BlockingRecv error: recv got EOF from 127.0.253.194:51733 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 12 16:37:48 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.195@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:48 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.195@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:48.343549 17762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:48.338373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.195:45413)
0812 16:37:48.338574 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:48.338588 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:48.338656 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:48.338795 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:48.338796 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:48.338966 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:48.339019 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:48.339021 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:48.339763 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:48.339764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:48.340401 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:48.340404 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:48.340948 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:48.340950 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:48.341021 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:48.341561 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:48.341566 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:48.343488 (+ 1922us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":101,"threads_started":1}
W20260812 16:37:48.343789 17760 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:48.338444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:45413 (local address 127.0.253.254:36149)
0812 16:37:48.338517 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:48.338520 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:48.338654 (+ 134us) server_negotiation.cc:407] Connection header received
0812 16:37:48.338713 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:48.338714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:48.338727 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:48.338769 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:48.339186 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:48.339684 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:48.340514 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:48.340822 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:48.340909 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:48.343653 (+ 2744us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.195:45413: BlockingRecv error: recv got EOF from 127.0.253.195:45413 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 12 16:37:49 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.193@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:49 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.193@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:49.102566 17764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:49.097638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.193:33205)
0812 16:37:49.097861 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:49.097873 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:49.097945 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:49.098081 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:49.098082 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:49.098255 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:49.098308 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:49.098310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:49.098974 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:49.098975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:49.099631 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:49.099634 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:49.100001 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:49.100002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:49.100051 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:49.100616 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:49.100621 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:49.102506 (+ 1885us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":114,"threads_started":1}
W20260812 16:37:49.102741 17766 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:49.097746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:33205 (local address 127.0.253.254:36149)
0812 16:37:49.097866 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:49.097868 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:49.097926 (+ 58us) server_negotiation.cc:407] Connection header received
0812 16:37:49.098001 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:49.098002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:49.098012 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:49.098061 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:49.098378 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:49.098905 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:49.099735 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:49.099927 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:49.099982 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:49.102657 (+ 2675us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.193:33205: BlockingRecv error: recv got EOF from 127.0.253.193:33205 (error 108)
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":58,"threads_started":1}
Aug 12 16:37:49 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.194@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:49 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.194@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:49.223470 17769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:49.218562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.194:33993)
0812 16:37:49.218721 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:49.218732 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:49.218781 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:49.218960 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:49.218961 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:49.219267 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:49.219320 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:49.219323 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:49.219969 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:49.219970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:49.220588 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:49.220590 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:49.220957 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:49.220958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:49.221009 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:49.221469 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:49.221473 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:49.223369 (+ 1896us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":92,"threads_started":1}
W20260812 16:37:49.223522 17766 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:49.218741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:33993 (local address 127.0.253.254:36149)
0812 16:37:49.218802 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:49.218805 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:49.218814 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:37:49.218851 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:49.218852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:49.218862 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:49.218953 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:49.219400 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:49.219897 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:49.220658 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:49.220846 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:49.220904 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:49.223458 (+ 2554us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.194:33993: BlockingRecv error: recv got EOF from 127.0.253.194:33993 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 12 16:37:49 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.195@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:49 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.195@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:49.349153 17770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:49.344166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.195:42891)
0812 16:37:49.344449 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:49.344473 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:49.344597 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:49.344862 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:49.344863 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:49.345122 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:49.345174 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:49.345176 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:49.345930 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:49.345930 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:49.346455 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:49.346457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:49.346793 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:49.346794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:49.346836 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:49.347370 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:49.347375 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:49.349094 (+ 1719us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":90,"threads_started":1}
W20260812 16:37:49.349270 17766 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:49.344318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:42891 (local address 127.0.253.254:36149)
0812 16:37:49.344380 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:49.344382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:49.344596 (+ 214us) server_negotiation.cc:407] Connection header received
0812 16:37:49.344699 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:49.344700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:49.344719 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:49.344785 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:49.345294 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:49.345846 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:49.346530 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:49.346725 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:49.346776 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:49.349194 (+ 2418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.195:42891: BlockingRecv error: recv got EOF from 127.0.253.195:42891 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 12 16:37:50 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.193@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:50 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.193@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:50.108533 17775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:50.103156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.193:45215)
0812 16:37:50.103321 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:50.103334 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:50.103399 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:50.103633 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:50.103633 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:50.103818 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:50.103871 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:50.103872 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:50.104575 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:50.104576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:50.105122 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:50.105124 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:50.105533 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:50.105535 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:50.105589 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:50.106162 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:50.106168 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:50.108471 (+ 2303us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":64,"threads_started":1}
W20260812 16:37:50.108768 17776 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:50.103323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:45215 (local address 127.0.253.254:36149)
0812 16:37:50.103447 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:50.103449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:50.103459 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:37:50.103492 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:50.103493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:50.103504 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:50.103605 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:50.103951 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:50.104473 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:50.105228 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:50.105442 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:50.105512 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:50.108669 (+ 3157us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.193:45215: BlockingRecv error: recv got EOF from 127.0.253.193:45215 (error 108)
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":69,"threads_started":1}
Aug 12 16:37:50 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.194@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:50 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.194@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:50.229096 17778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:50.224042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.194:53629)
0812 16:37:50.224220 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:50.224231 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:50.224288 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:50.224486 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:50.224488 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:50.224700 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:50.224751 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:50.224754 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:50.225467 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:50.225468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:50.226037 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:50.226039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:50.226392 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:50.226393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:50.226436 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:50.226969 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:50.226975 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:50.229033 (+ 2058us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":111,"threads_started":1}
W20260812 16:37:50.229247 17776 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:50.224182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:53629 (local address 127.0.253.254:36149)
0812 16:37:50.224229 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:50.224231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:50.224281 (+ 50us) server_negotiation.cc:407] Connection header received
0812 16:37:50.224347 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:50.224348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:50.224359 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:50.224418 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:50.224863 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:50.225388 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:50.226112 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:50.226324 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:50.226379 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:50.229158 (+ 2779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.194:53629: BlockingRecv error: recv got EOF from 127.0.253.194:53629 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 12 16:37:50 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.195@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:50 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.195@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:50.354612 17779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:50.349747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.195:37339)
0812 16:37:50.349891 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:50.349903 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:50.349976 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:50.350119 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:50.350120 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:50.350310 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:50.350360 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:50.350362 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:50.351022 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:50.351022 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:50.351611 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:50.351613 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:50.351973 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:50.351974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:50.352015 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:50.352616 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:50.352621 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:50.354554 (+ 1933us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":89,"threads_started":1}
W20260812 16:37:50.354771 17776 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:50.349830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:37339 (local address 127.0.253.254:36149)
0812 16:37:50.349905 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:50.349907 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:50.349972 (+ 65us) server_negotiation.cc:407] Connection header received
0812 16:37:50.350032 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:50.350032 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:50.350045 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:50.350105 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:50.350433 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:50.350950 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:50.351707 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:50.351898 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:50.351965 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:50.354683 (+ 2718us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.195:37339: BlockingRecv error: recv got EOF from 127.0.253.195:37339 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 12 16:37:51 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.193@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:51 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.193@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:51.114024 17780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:51.109108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.193:33177)
0812 16:37:51.109278 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:51.109296 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:51.109351 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:51.109501 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:51.109502 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:51.109694 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:51.109750 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:51.109752 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:51.110476 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:51.110477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:51.111024 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:51.111025 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:51.111478 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:51.111479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:51.111546 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:51.112079 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:51.112085 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:51.113964 (+ 1879us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":107,"threads_started":1}
W20260812 16:37:51.114178 17781 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:51.109197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:33177 (local address 127.0.253.254:36149)
0812 16:37:51.109360 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:51.109362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:51.109372 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:37:51.109399 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:51.109399 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:51.109411 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:51.109481 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:51.109861 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:51.110398 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:51.111171 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:51.111371 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:51.111431 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:51.114076 (+ 2645us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.193:33177: BlockingRecv error: recv got EOF from 127.0.253.193:33177 (error 108)
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":93,"threads_started":1}
Aug 12 16:37:51 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.194@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:51 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.194@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:51.234630 17782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:51.229616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.194:33667)
0812 16:37:51.229959 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:51.229970 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:51.230034 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:51.230172 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:51.230172 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:51.230346 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:51.230400 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:51.230402 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:51.231185 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:51.231186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:51.231709 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:51.231711 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:51.232051 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:51.232052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:51.232094 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:51.232611 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:51.232616 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:51.234570 (+ 1954us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":88,"threads_started":1}
W20260812 16:37:51.234772 17781 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:51.229686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:33667 (local address 127.0.253.254:36149)
0812 16:37:51.229956 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:51.229958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:51.230042 (+ 84us) server_negotiation.cc:407] Connection header received
0812 16:37:51.230077 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:51.230078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:51.230093 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:51.230147 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:51.230506 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:51.231032 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:51.231786 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:51.231978 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:51.232040 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:51.234683 (+ 2643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.194:33667: BlockingRecv error: recv got EOF from 127.0.253.194:33667 (error 108)
Metrics: {"server-negotiator.queue_time_us":237}
Aug 12 16:37:51 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.195@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:51 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.195@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:51.360453 17783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:51.355334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.195:56069)
0812 16:37:51.355465 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:51.355478 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:51.355544 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:51.355700 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:51.355700 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:51.355869 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:51.355923 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:51.355924 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:51.356595 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:51.356597 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:51.357142 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:51.357144 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:51.357666 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:51.357667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:51.357716 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:51.358335 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:51.358340 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:51.360357 (+ 2017us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":61,"threads_started":1}
W20260812 16:37:51.360553 17781 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:51.355404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:56069 (local address 127.0.253.254:36149)
0812 16:37:51.355499 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:51.355501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:51.355534 (+ 33us) server_negotiation.cc:407] Connection header received
0812 16:37:51.355604 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:51.355605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:51.355619 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:51.355680 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:51.355984 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:51.356482 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:51.357251 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:51.357524 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:51.357611 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:51.360481 (+ 2870us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.195:56069: BlockingRecv error: recv got EOF from 127.0.253.195:56069 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 12 16:37:52 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.193@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:52 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.193@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:52.120105 17785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:52.114589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.193:56625)
0812 16:37:52.114760 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:52.114769 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:52.114813 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:52.114991 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:52.114991 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:52.115191 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:52.115247 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:52.115249 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:52.115949 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:52.115951 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:52.116504 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:52.116506 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:52.116915 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:52.116916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:52.116970 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:52.117576 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:52.117584 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:52.120027 (+ 2443us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":66,"threads_started":1}
W20260812 16:37:52.120308 17786 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:52.114727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:56625 (local address 127.0.253.254:36149)
0812 16:37:52.114870 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:52.114872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:52.114879 (+ 7us) server_negotiation.cc:407] Connection header received
0812 16:37:52.114902 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:52.114903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:52.114913 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:52.114976 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:52.115314 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:52.115827 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:52.116597 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:52.116816 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:52.116875 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:52.120220 (+ 3345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.193:56625: BlockingRecv error: recv got EOF from 127.0.253.193:56625 (error 108)
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":84,"threads_started":1}
Aug 12 16:37:52 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.194@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:52 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.194@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:52.240286 17789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:52.235268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.194:51861)
0812 16:37:52.235421 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:52.235441 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:52.235522 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:52.235673 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:52.235674 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:52.235875 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:52.235927 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:52.235929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:52.236773 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:52.236774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:52.237324 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:52.237327 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:52.237654 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:52.237655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:52.237697 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:52.238238 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:52.238243 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:52.240216 (+ 1973us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":90,"threads_started":1}
W20260812 16:37:52.240389 17786 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:52.235364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:51861 (local address 127.0.253.254:36149)
0812 16:37:52.235422 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:52.235424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:52.235501 (+ 77us) server_negotiation.cc:407] Connection header received
0812 16:37:52.235579 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:52.235580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:52.235592 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:52.235650 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:52.236032 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:52.236654 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:52.237403 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:52.237588 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:52.237638 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:52.240320 (+ 2682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.194:51861: BlockingRecv error: recv got EOF from 127.0.253.194:51861 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 12 16:37:52 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.195@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:52 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.195@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:52.365913 17790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:52.361057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.195:37253)
0812 16:37:52.361181 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:52.361195 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:52.361264 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:52.361396 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:52.361397 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:52.361572 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:52.361622 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:52.361623 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:52.362308 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:52.362311 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:52.362942 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:52.362944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:52.363384 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:52.363385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:52.363448 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:52.363983 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:52.363989 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:52.365840 (+ 1851us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":61,"threads_started":1}
W20260812 16:37:52.366007 17786 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:52.361125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:37253 (local address 127.0.253.254:36149)
0812 16:37:52.361237 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:52.361239 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:52.361250 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:37:52.361326 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:52.361327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:52.361341 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:52.361391 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:52.361693 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:52.362205 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:52.363023 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:52.363302 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:52.363382 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:52.365940 (+ 2558us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.195:37253: BlockingRecv error: recv got EOF from 127.0.253.195:37253 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 12 16:37:53 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.193@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:53 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.193@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:53.140061 17791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:53.120858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.193:52945)
0812 16:37:53.121033 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:53.121045 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:53.121101 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:53.121226 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:53.121227 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:53.121403 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:53.121451 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:53.121453 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:53.122144 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:53.122145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:53.122687 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:53.122689 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:53.123049 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:53.123050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:53.123120 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:53.137658 (+ 14538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:53.137682 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:53.139949 (+ 2267us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":100,"threads_started":1}
W20260812 16:37:53.140123 17794 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:53.120944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:52945 (local address 127.0.253.254:36149)
0812 16:37:53.121087 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:53.121089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:53.121099 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:37:53.121150 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:53.121151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:53.121162 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:53.121219 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:53.121553 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:53.122069 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:53.122782 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:53.122977 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:53.123034 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:53.140049 (+ 17015us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.193:52945: BlockingRecv error: recv got EOF from 127.0.253.193:52945 (error 108)
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":93,"threads_started":1}
Aug 12 16:37:53 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.194@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:53 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.194@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:53.245630 17795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:53.240841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.194:59733)
0812 16:37:53.241036 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:53.241047 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:53.241110 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:53.241279 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:53.241279 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:53.241453 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:53.241503 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:53.241505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:53.242203 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:53.242204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:53.242731 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:53.242733 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:53.243136 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:53.243137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:53.243188 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:53.243658 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:53.243663 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:53.245564 (+ 1901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":122,"threads_started":1}
W20260812 16:37:53.245893 17794 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:53.240905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:59733 (local address 127.0.253.254:36149)
0812 16:37:53.240978 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:53.240981 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:53.241171 (+ 190us) server_negotiation.cc:407] Connection header received
0812 16:37:53.241199 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:53.241200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:53.241213 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:53.241260 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:53.241605 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:53.242128 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:53.242824 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:53.243017 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:53.243117 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:53.245826 (+ 2709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.194:59733: BlockingRecv error: recv got EOF from 127.0.253.194:59733 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 12 16:37:53 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.195@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:53 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.195@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:53.371797 17797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:53.366455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.195:60837)
0812 16:37:53.366674 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:53.366686 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:53.366782 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:53.366954 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:53.366955 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:53.367217 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:53.367294 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:53.367297 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:53.368089 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:53.368090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:53.368790 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:53.368793 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:53.369168 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:53.369169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:53.369243 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:53.369718 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:53.369723 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:53.371687 (+ 1964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":94,"threads_started":1}
W20260812 16:37:53.371831 17794 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:53.366559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:60837 (local address 127.0.253.254:36149)
0812 16:37:53.366657 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:53.366659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:53.366761 (+ 102us) server_negotiation.cc:407] Connection header received
0812 16:37:53.366855 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:53.366856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:53.366873 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:53.366947 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:53.367387 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:53.367979 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:53.368874 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:53.369079 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:53.369165 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:53.371765 (+ 2600us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.195:60837: BlockingRecv error: recv got EOF from 127.0.253.195:60837 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 12 16:37:54 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.193@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:54 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.193@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:54.145161 17798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:54.140560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.193:48225)
0812 16:37:54.140699 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:54.140713 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:54.140811 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:54.140937 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:54.140937 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:54.141117 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:54.141170 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:54.141173 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:54.141811 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:54.141812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:54.142326 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:54.142327 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:54.142673 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:54.142674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:54.142712 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:54.143216 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:54.143221 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:54.145067 (+ 1846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":71,"threads_started":1}
W20260812 16:37:54.145232 17800 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:54.140649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:48225 (local address 127.0.253.254:36149)
0812 16:37:54.140809 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:54.140811 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:54.140820 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:37:54.140861 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:54.140861 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:54.140872 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:54.140923 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:54.141235 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:54.141737 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:54.142404 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:54.142598 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:54.142657 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:54.145163 (+ 2506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.193:48225: BlockingRecv error: recv got EOF from 127.0.253.193:48225 (error 108)
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":74,"threads_started":1}
Aug 12 16:37:54 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.194@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:54 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.194@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:54.251282 17801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:54.246155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.194:54351)
0812 16:37:54.246324 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:54.246335 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:54.246393 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:54.246544 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:54.246545 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:54.246721 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:54.246774 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:54.246776 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:54.247487 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:54.247488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:54.248073 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:54.248075 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:54.248583 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:54.248584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:54.248656 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:54.249204 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:54.249210 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:54.251196 (+ 1986us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":104,"threads_started":1}
W20260812 16:37:54.251415 17800 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:54.246275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:54351 (local address 127.0.253.254:36149)
0812 16:37:54.246326 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:54.246328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:54.246389 (+ 61us) server_negotiation.cc:407] Connection header received
0812 16:37:54.246456 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:54.246456 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:54.246468 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:54.246511 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:54.246844 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:54.247366 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:54.248207 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:54.248493 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:54.248570 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:54.251313 (+ 2743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.194:54351: BlockingRecv error: recv got EOF from 127.0.253.194:54351 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 12 16:37:54 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.195@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:54 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.195@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:54.377110 17802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:54.372358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.195:57779)
0812 16:37:54.372535 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:54.372550 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:54.372622 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:54.372804 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:54.372805 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:54.372978 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:54.373028 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:54.373030 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:54.373674 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:54.373675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:54.374226 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:54.374228 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:54.374550 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:54.374551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:54.374597 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:54.375175 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:54.375191 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:54.377028 (+ 1837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":76,"threads_started":1}
W20260812 16:37:54.377177 17800 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:54.372505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:57779 (local address 127.0.253.254:36149)
0812 16:37:54.372575 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:54.372577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:54.372605 (+ 28us) server_negotiation.cc:407] Connection header received
0812 16:37:54.372700 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:54.372701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:54.372713 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:54.372782 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:54.373091 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:54.373586 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:54.374299 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:54.374477 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:54.374527 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:54.377127 (+ 2600us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.195:57779: BlockingRecv error: recv got EOF from 127.0.253.195:57779 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 12 16:37:55 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.193@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:55 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.193@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:55.151051 17808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:55.145778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.193:53807)
0812 16:37:55.145962 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:55.145977 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:55.146053 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:55.146292 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:55.146293 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:55.146551 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:55.146609 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:55.146611 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:55.147483 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:55.147484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:55.148038 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:55.148040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:55.148423 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:55.148425 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:55.148493 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:55.148981 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:55.148986 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:55.150939 (+ 1953us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":74,"threads_started":1}
W20260812 16:37:55.151137 17810 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:55.146018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:53807 (local address 127.0.253.254:36149)
0812 16:37:55.146159 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:55.146161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:55.146171 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:37:55.146196 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:55.146197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:55.146208 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:55.146271 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:55.146705 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:55.147386 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:55.148136 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:55.148345 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:55.148424 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:55.151027 (+ 2603us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.193:53807: BlockingRecv error: recv got EOF from 127.0.253.193:53807 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":77,"threads_started":1}
Aug 12 16:37:55 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.194@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:55 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.194@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:55.257211 17814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:55.251821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.194:55869)
0812 16:37:55.251997 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:55.252008 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:55.252075 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:55.252282 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:55.252283 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:55.252477 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:55.252523 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:55.252526 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:55.253207 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:55.253208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:55.253764 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:55.253766 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:55.254145 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:55.254146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:55.254205 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:55.254770 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:55.254775 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:55.257106 (+ 2331us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":97,"threads_started":1}
W20260812 16:37:55.257277 17810 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:55.251953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:55869 (local address 127.0.253.254:36149)
0812 16:37:55.252039 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:55.252041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:55.252054 (+ 13us) server_negotiation.cc:407] Connection header received
0812 16:37:55.252183 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:55.252184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:55.252197 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:55.252258 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:55.252601 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:55.253113 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:55.253861 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:55.254063 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:55.254126 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:55.257227 (+ 3101us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.194:55869: BlockingRecv error: recv got EOF from 127.0.253.194:55869 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 12 16:37:55 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.195@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:55 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.195@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:55.382841 17816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:55.377648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.195:48089)
0812 16:37:55.377883 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:55.377903 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:55.377988 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:55.378158 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:55.378159 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:55.378366 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:55.378413 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:55.378415 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:55.379288 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:55.379289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:55.379838 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:55.379840 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:55.380223 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:55.380224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:55.380295 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:55.380831 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:55.380836 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:55.382705 (+ 1869us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":137,"threads_started":1}
W20260812 16:37:55.382874 17810 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:55.377794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:48089 (local address 127.0.253.254:36149)
0812 16:37:55.377912 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:55.377914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:55.377981 (+ 67us) server_negotiation.cc:407] Connection header received
0812 16:37:55.378081 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:55.378082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:55.378096 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:55.378139 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:55.378505 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:55.379203 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:55.379932 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:55.380145 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:55.380224 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:55.382802 (+ 2578us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.195:48089: BlockingRecv error: recv got EOF from 127.0.253.195:48089 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 12 16:37:56 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.193@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:56 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.193@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:56.156744 17817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:56.151647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.193:36521)
0812 16:37:56.151823 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:56.151836 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:56.151919 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:56.152068 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:56.152068 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:56.152257 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:56.152311 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:56.152313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:56.153059 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:56.153060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:56.153589 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:56.153590 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:56.154024 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:56.154025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:56.154098 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:56.154730 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:56.154735 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:56.156680 (+ 1945us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":65,"threads_started":1}
W20260812 16:37:56.156855 17820 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:56.151766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:36521 (local address 127.0.253.254:36149)
0812 16:37:56.151936 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:56.151938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:56.151947 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:37:56.151973 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:56.151973 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:56.151983 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:56.152046 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:56.152380 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:56.152988 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:56.153710 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:56.153945 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:56.154010 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:56.156777 (+ 2767us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.193:36521: BlockingRecv error: recv got EOF from 127.0.253.193:36521 (error 108)
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":78,"threads_started":1}
Aug 12 16:37:56 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.194@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:56 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.194@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:56.263216 17821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:56.257851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.194:41855)
0812 16:37:56.258010 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:56.258022 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:56.258082 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:56.258251 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:56.258252 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:56.258436 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:56.258568 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:56.258570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:56.259323 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:56.259325 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:56.259937 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:56.259940 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:56.260377 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:56.260378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:56.260433 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:56.260921 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:56.260926 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:56.263155 (+ 2229us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":76,"threads_started":1}
W20260812 16:37:56.263331 17820 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:56.257930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:41855 (local address 127.0.253.254:36149)
0812 16:37:56.258019 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:56.258022 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:56.258068 (+ 46us) server_negotiation.cc:407] Connection header received
0812 16:37:56.258165 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:56.258166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:56.258178 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:56.258221 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:56.258674 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:56.259218 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:56.260081 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:56.260283 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:56.260341 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:56.263252 (+ 2911us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.194:41855: BlockingRecv error: recv got EOF from 127.0.253.194:41855 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 12 16:37:56 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.195@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:56 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.195@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:56.387966 17822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:56.383341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.195:38493)
0812 16:37:56.383490 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:56.383502 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:56.383575 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:56.383708 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:56.383709 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:56.383885 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:56.383933 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:56.383934 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:56.384579 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:56.384580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:56.385136 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:56.385137 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:56.385456 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:56.385457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:56.385502 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:56.385978 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:56.385982 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:56.387903 (+ 1921us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":89,"threads_started":1}
W20260812 16:37:56.388068 17820 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:56.383419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:38493 (local address 127.0.253.254:36149)
0812 16:37:56.383498 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:56.383500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:56.383568 (+ 68us) server_negotiation.cc:407] Connection header received
0812 16:37:56.383620 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:56.383621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:56.383633 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:56.383687 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:56.383997 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:56.384497 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:56.385213 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:56.385391 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:56.385441 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:56.388000 (+ 2559us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.195:38493: BlockingRecv error: recv got EOF from 127.0.253.195:38493 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 12 16:37:57 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.193@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:57 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.193@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:57.162505 17823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:57.157323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.193:36401)
0812 16:37:57.157501 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:57.157514 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:57.157588 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:57.157753 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:57.157754 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:57.157933 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:57.158035 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:57.158038 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:57.158977 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:57.158979 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:57.159538 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:57.159540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:57.159923 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:57.159924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:57.159990 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:57.160522 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:57.160526 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:57.162445 (+ 1919us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":94,"threads_started":1}
W20260812 16:37:57.162662 17824 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:57.157385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:36401 (local address 127.0.253.254:36149)
0812 16:37:57.157540 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:57.157542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:57.157571 (+ 29us) server_negotiation.cc:407] Connection header received
0812 16:37:57.157645 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:57.157645 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:57.157656 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:57.157713 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:57.158167 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:57.158884 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:57.159626 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:57.159831 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:57.159920 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:57.162565 (+ 2645us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.193:36401: BlockingRecv error: recv got EOF from 127.0.253.193:36401 (error 108)
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":89,"threads_started":1}
Aug 12 16:37:57 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.194@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:57 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.194@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:57.268762 17829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:57.263795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.194:44031)
0812 16:37:57.263953 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:57.263965 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:57.264048 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:57.264219 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:57.264220 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:57.264430 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:57.264477 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:57.264479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:57.265267 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:57.265269 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:57.265934 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:57.265937 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:57.266318 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:57.266319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:57.266382 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:57.266836 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:57.266841 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:57.268658 (+ 1817us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":90,"threads_started":1}
W20260812 16:37:57.268854 17824 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:57.263949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:44031 (local address 127.0.253.254:36149)
0812 16:37:57.264030 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:57.264033 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:57.264043 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:37:57.264122 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:57.264123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:57.264143 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:57.264210 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:57.264566 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:57.265178 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:57.266020 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:57.266225 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:57.266315 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:57.268789 (+ 2474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.194:44031: BlockingRecv error: recv got EOF from 127.0.253.194:44031 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 12 16:37:57 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.195@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:57 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.195@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:57.393533 17830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:57.388464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.195:49499)
0812 16:37:57.388666 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:57.388678 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:57.388746 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:57.388876 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:57.388877 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:57.389054 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:57.389108 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:57.389110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:57.389795 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:57.389796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:57.390371 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:57.390372 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:57.390766 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:57.390767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:57.390820 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:57.391357 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:57.391362 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:57.393456 (+ 2094us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":150,"threads_started":1}
W20260812 16:37:57.393663 17824 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:57.388541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:49499 (local address 127.0.253.254:36149)
0812 16:37:57.388601 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:57.388603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:57.388745 (+ 142us) server_negotiation.cc:407] Connection header received
0812 16:37:57.388791 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:57.388792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:57.388805 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:57.388857 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:57.389187 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:57.389720 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:57.390455 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:57.390662 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:57.390723 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:57.393577 (+ 2854us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.195:49499: BlockingRecv error: recv got EOF from 127.0.253.195:49499 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 12 16:37:58 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.193@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:58 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.193@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:58.171129 17831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:58.162988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.193:51907)
0812 16:37:58.163193 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:58.163216 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:58.163275 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:58.163415 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:58.163416 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:58.163590 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:58.163654 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:58.163655 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:58.164365 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:58.164366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:58.164912 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:58.164913 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:58.165271 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:58.165271 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:58.165314 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:58.168999 (+ 3685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:58.169004 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:58.170998 (+ 1994us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":141,"threads_started":1}
W20260812 16:37:58.171288 17832 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:58.163127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:51907 (local address 127.0.253.254:36149)
0812 16:37:58.163251 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:58.163252 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:58.163262 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:37:58.163334 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:58.163335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:58.163346 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:58.163394 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:58.163763 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:58.164292 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:58.164991 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:58.165200 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:58.165254 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:58.171188 (+ 5934us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.193:51907: BlockingRecv error: recv got EOF from 127.0.253.193:51907 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":55,"threads_started":1}
Aug 12 16:37:58 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.194@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:58 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.194@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:58.274173 17834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:58.269279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.194:42653)
0812 16:37:58.269446 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:58.269460 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:58.269526 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:58.269658 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:58.269659 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:58.269844 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:58.269895 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:58.269897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:58.270596 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:58.270597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:58.271178 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:58.271180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:58.271580 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:58.271581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:58.271628 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:58.272155 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:58.272160 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:58.274102 (+ 1942us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":105,"threads_started":1}
W20260812 16:37:58.274326 17832 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:58.269369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:42653 (local address 127.0.253.254:36149)
0812 16:37:58.269432 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:58.269434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:58.269512 (+ 78us) server_negotiation.cc:407] Connection header received
0812 16:37:58.269573 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:58.269573 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:58.269584 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:58.269635 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:58.269961 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:58.270468 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:58.271279 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:58.271484 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:58.271560 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:58.274240 (+ 2680us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.194:42653: BlockingRecv error: recv got EOF from 127.0.253.194:42653 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 12 16:37:58 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.195@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:58 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.195@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:58.399576 17835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:58.394309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.195:59937)
0812 16:37:58.394458 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:58.394468 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:58.394529 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:58.394681 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:58.394682 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:58.394868 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:58.394920 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:58.394922 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:58.395626 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:58.395627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:58.396346 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:58.396350 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:58.396716 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:58.396718 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:58.396779 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:58.397297 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:58.397303 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:58.399473 (+ 2170us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":83,"threads_started":1}
W20260812 16:37:58.399628 17832 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:58.394407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:59937 (local address 127.0.253.254:36149)
0812 16:37:58.394474 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:58.394476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:58.394515 (+ 39us) server_negotiation.cc:407] Connection header received
0812 16:37:58.394589 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:58.394590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:58.394605 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:58.394661 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:58.394986 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:58.395554 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:58.396416 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:58.396628 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:58.396709 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:58.399561 (+ 2852us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.195:59937: BlockingRecv error: recv got EOF from 127.0.253.195:59937 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 12 16:37:59 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.193@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:59 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.193@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:59.176544 17836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:59.171755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.193:36709)
0812 16:37:59.171914 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:59.171928 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:59.171986 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:59.172145 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:59.172146 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:59.172322 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:59.172376 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:59.172378 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:59.173096 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:59.173097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:59.173641 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:59.173642 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:59.174014 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:59.174015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:59.174058 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:59.174630 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:59.174634 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:59.176483 (+ 1849us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":101,"threads_started":1}
W20260812 16:37:59.176684 17841 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:59.171830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:36709 (local address 127.0.253.254:36149)
0812 16:37:59.171960 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:59.171962 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:59.171975 (+ 13us) server_negotiation.cc:407] Connection header received
0812 16:37:59.172045 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:59.172046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:59.172057 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:59.172118 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:59.172482 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:59.172996 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:59.173738 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:59.173937 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:59.173993 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:59.176593 (+ 2600us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.193:36709: BlockingRecv error: recv got EOF from 127.0.253.193:36709 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":79,"threads_started":1}
Aug 12 16:37:59 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.194@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:37:59 dist-test-slave-w4v5 krb5kdc[15915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.253.194@KRBTEST.COM for oryx/127.0.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:37:59.279794 17843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:59.274772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36149 (local address 127.0.253.194:58307)
0812 16:37:59.274953 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:59.274965 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:59.275033 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:59.275184 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:59.275184 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:59.275360 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:59.275411 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:59.275413 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:59.276065 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:59.276066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:59.276650 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:59.276652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:59.277012 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:59.277012 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:59.277052 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:59.277629 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:37:59.277635 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:59.279683 (+ 2048us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.253.254:36149: Server oryx/127.0.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":94,"threads_started":1}
W20260812 16:37:59.279955 17841 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:59.274906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:58307 (local address 127.0.253.254:36149)
0812 16:37:59.274986 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:59.274988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:59.275017 (+ 29us) server_negotiation.cc:407] Connection header received
0812 16:37:59.275115 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:59.275116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:59.275127 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:59.275179 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:59.275475 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:59.275978 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:59.276741 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:59.276939 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:59.276998 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:59.279879 (+ 2881us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.253.194:58307: BlockingRecv error: recv got EOF from 127.0.253.194:58307 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 16:37:59.312361 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 17010
I20260812 16:37:59.331522 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 17272
I20260812 16:37:59.345425 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 17499
I20260812 16:37:59.353483 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 16870
2026-08-12T16:37:59Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17876 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-12T16:37:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:37:59Z Disabled control of system clock
I20260812 16:37:59.393385 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:45951
--webserver_interface=127.0.253.254
--webserver_port=0
--builtin_ntp_servers=127.0.253.212:39713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:45951
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 16:37:59.480746 17848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:59.480981 17848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:59.481015 17848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:59.482362 17848 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:37:59.482456 17848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:59.482532 17848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:59.482579 17848 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:37:59.482609 17848 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:37:59.484297 17848 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:39713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:45951
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:45951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17848
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:59.484603 17848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:59.484921 17848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:59.490109 17855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:59.490162 17856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:59.490141 17859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:59.490530 17848 server_base.cc:1034] running on GCE node
I20260812 16:37:59.490836 17848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:59.491225 17848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:59.492374 17848 hybrid_clock.cc:648] HybridClock initialized: now 1786552679492352 us; error 43 us; skew 500 ppm
I20260812 16:37:59.493785 17848 webserver.cc:460] Webserver started at http://127.0.253.254:40603/ using document root <none> and password file <none>
I20260812 16:37:59.494060 17848 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:59.494109 17848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:59.494289 17848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:59.495487 17848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "c9581aaf67c04e07b0b84f6fdfa85849"
format_stamp: "Formatted at 2026-08-12 16:37:59 on dist-test-slave-w4v5"
server_key: "132fa5f47d9d9ac4b43b8eb16659c4db"
server_key_iv: "1473e02bd40995202c14bc0eb1b30602"
server_key_version: "encryptionkey@0"
I20260812 16:37:59.495867 17848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "c9581aaf67c04e07b0b84f6fdfa85849"
format_stamp: "Formatted at 2026-08-12 16:37:59 on dist-test-slave-w4v5"
server_key: "132fa5f47d9d9ac4b43b8eb16659c4db"
server_key_iv: "1473e02bd40995202c14bc0eb1b30602"
server_key_version: "encryptionkey@0"
I20260812 16:37:59.497226 17848 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:37:59.498001 17867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:59.498261 17848 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:59.498360 17848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "c9581aaf67c04e07b0b84f6fdfa85849"
format_stamp: "Formatted at 2026-08-12 16:37:59 on dist-test-slave-w4v5"
server_key: "132fa5f47d9d9ac4b43b8eb16659c4db"
server_key_iv: "1473e02bd40995202c14bc0eb1b30602"
server_key_version: "encryptionkey@0"
I20260812 16:37:59.498420 17848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:59.513799 17848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:59.514091 17848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:59.514197 17848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:59.518512 17848 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:45951
I20260812 16:37:59.518548 17963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:45951 every 8 connection(s)
I20260812 16:37:59.518918 17848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:37:59.519702 17964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:37:59.521009 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 17848
I20260812 16:37:59.521092 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:37:59.521296 1015 external_mini_cluster.cc:1428] Setting key 39058fde57b7b0ee9e11a49b4c73eef1
I20260812 16:37:59.522782 17964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9581aaf67c04e07b0b84f6fdfa85849: Bootstrap starting.
I20260812 16:37:59.523518 17964 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9581aaf67c04e07b0b84f6fdfa85849: Neither blocks nor log segments found. Creating new log.
I20260812 16:37:59.523834 17964 log.cc:824] T 00000000000000000000000000000000 P c9581aaf67c04e07b0b84f6fdfa85849: Log is configured to *not* fsync() on all Append() calls
I20260812 16:37:59.524637 17964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9581aaf67c04e07b0b84f6fdfa85849: No bootstrap required, opened a new log
I20260812 16:37:59.527212 17964 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9581aaf67c04e07b0b84f6fdfa85849 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9581aaf67c04e07b0b84f6fdfa85849" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 45951 } }
I20260812 16:37:59.527410 17969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:59.521894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43894 (local address 127.0.253.254:45951)
0812 16:37:59.522067 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:59.522074 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:59.522098 (+ 24us) server_negotiation.cc:407] Connection header received
0812 16:37:59.522580 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:59.522587 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:59.522652 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:59.522783 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:59.523049 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:59.524108 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:59.524977 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:59.525616 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:59.525759 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:59.526460 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:59.526466 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:59.526467 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:37:59.526733 (+ 266us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:59.526789 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:37:59.526809 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:59.526893 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:59.527021 (+ 128us) server_negotiation.cc:299] Negotiation successful
0812 16:37:59.527146 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":64,"threads_started":1}
I20260812 16:37:59.527419 17964 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9581aaf67c04e07b0b84f6fdfa85849 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:37:59.527518 17964 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9581aaf67c04e07b0b84f6fdfa85849 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9581aaf67c04e07b0b84f6fdfa85849, State: Initialized, Role: FOLLOWER
I20260812 16:37:59.527668 17964 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9581aaf67c04e07b0b84f6fdfa85849 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9581aaf67c04e07b0b84f6fdfa85849" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 45951 } }
I20260812 16:37:59.527778 17964 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9581aaf67c04e07b0b84f6fdfa85849 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:37:59.527824 17964 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9581aaf67c04e07b0b84f6fdfa85849 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:37:59.527874 17964 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9581aaf67c04e07b0b84f6fdfa85849 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:37:59.528551 17964 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9581aaf67c04e07b0b84f6fdfa85849 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9581aaf67c04e07b0b84f6fdfa85849" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 45951 } }
I20260812 16:37:59.528679 17964 leader_election.cc:302] T 00000000000000000000000000000000 P c9581aaf67c04e07b0b84f6fdfa85849 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9581aaf67c04e07b0b84f6fdfa85849; no voters:
I20260812 16:37:59.528877 17964 leader_election.cc:288] T 00000000000000000000000000000000 P c9581aaf67c04e07b0b84f6fdfa85849 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:37:59.528959 17971 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9581aaf67c04e07b0b84f6fdfa85849 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:37:59.529166 17971 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9581aaf67c04e07b0b84f6fdfa85849 [term 1 LEADER]: Becoming Leader. State: Replica: c9581aaf67c04e07b0b84f6fdfa85849, State: Running, Role: LEADER
I20260812 16:37:59.529206 17964 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9581aaf67c04e07b0b84f6fdfa85849 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:37:59.529261 17971 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9581aaf67c04e07b0b84f6fdfa85849 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9581aaf67c04e07b0b84f6fdfa85849" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 45951 } }
I20260812 16:37:59.529680 17973 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9581aaf67c04e07b0b84f6fdfa85849 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9581aaf67c04e07b0b84f6fdfa85849" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9581aaf67c04e07b0b84f6fdfa85849" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 45951 } } }
I20260812 16:37:59.529784 17973 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9581aaf67c04e07b0b84f6fdfa85849 [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:59.529961 17975 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9581aaf67c04e07b0b84f6fdfa85849 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9581aaf67c04e07b0b84f6fdfa85849. Latest consensus state: current_term: 1 leader_uuid: "c9581aaf67c04e07b0b84f6fdfa85849" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9581aaf67c04e07b0b84f6fdfa85849" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 45951 } } }
I20260812 16:37:59.530037 17975 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9581aaf67c04e07b0b84f6fdfa85849 [sys.catalog]: This master's current role is: LEADER
I20260812 16:37:59.530134 17983 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:37:59.530691 17983 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:37:59.532323 17983 catalog_manager.cc:1347] Generated new cluster ID: b3a46b68dfc64ab59028256d35ca56cc
I20260812 16:37:59.532392 17983 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:37:59.545894 17983 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:37:59.546470 17983 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:37:59.554020 17983 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9581aaf67c04e07b0b84f6fdfa85849: Generated new TSK 0
I20260812 16:37:59.554231 17983 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:37:59.558259 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.193:0
--local_ip_for_outbound_sockets=127.0.253.193
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:45951
--builtin_ntp_servers=127.0.253.212:39713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 16:37:59.646831 18000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:59.647092 18000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:59.647176 18000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:59.648372 18000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:59.648443 18000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:59.648487 18000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.193
I20260812 16:37:59.649905 18000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:39713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:45951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18000
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.193
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:59.650172 18000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:59.650511 18000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:59.656735 18015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:59.656759 18010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:59.656900 18009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:59.657158 18000 server_base.cc:1034] running on GCE node
I20260812 16:37:59.657387 18000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:59.657631 18000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:59.658826 18000 hybrid_clock.cc:648] HybridClock initialized: now 1786552679658812 us; error 30 us; skew 500 ppm
I20260812 16:37:59.660283 18000 webserver.cc:460] Webserver started at http://127.0.253.193:46225/ using document root <none> and password file <none>
I20260812 16:37:59.660576 18000 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:59.660653 18000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:59.660796 18000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:59.662071 18000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/data/instance:
uuid: "6445f670cea147a586a4b1e4cbcb34f2"
format_stamp: "Formatted at 2026-08-12 16:37:59 on dist-test-slave-w4v5"
server_key: "63945159048d3e80c8f2a373255375fc"
server_key_iv: "102819fe3d70bd1fedc9740d9968867c"
server_key_version: "encryptionkey@0"
I20260812 16:37:59.662456 18000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance:
uuid: "6445f670cea147a586a4b1e4cbcb34f2"
format_stamp: "Formatted at 2026-08-12 16:37:59 on dist-test-slave-w4v5"
server_key: "63945159048d3e80c8f2a373255375fc"
server_key_iv: "102819fe3d70bd1fedc9740d9968867c"
server_key_version: "encryptionkey@0"
I20260812 16:37:59.663918 18000 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:37:59.664676 18023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:59.664850 18000 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 16:37:59.664959 18000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/wal
uuid: "6445f670cea147a586a4b1e4cbcb34f2"
format_stamp: "Formatted at 2026-08-12 16:37:59 on dist-test-slave-w4v5"
server_key: "63945159048d3e80c8f2a373255375fc"
server_key_iv: "102819fe3d70bd1fedc9740d9968867c"
server_key_version: "encryptionkey@0"
I20260812 16:37:59.665035 18000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:59.684311 18000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:59.684725 18000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:59.684898 18000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:59.685158 18000 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:37:59.685518 18000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:59.685575 18000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:59.685619 18000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:59.685676 18000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:59.692186 18000 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.193:41657
I20260812 16:37:59.692230 18216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.193:41657 every 8 connection(s)
I20260812 16:37:59.692608 18000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
I20260812 16:37:59.696815 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 18000
I20260812 16:37:59.696928 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance
I20260812 16:37:59.697140 1015 external_mini_cluster.cc:1428] Setting key 49be7b732ea714aae2d889590f795fd6
I20260812 16:37:59.698045 17969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:59.693407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:37815 (local address 127.0.253.254:45951)
0812 16:37:59.693496 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:59.693500 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:59.693778 (+ 278us) server_negotiation.cc:407] Connection header received
0812 16:37:59.694277 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:59.694279 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:59.694289 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:59.694343 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:59.694996 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:59.695575 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:59.696512 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:59.696696 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:59.696781 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:59.697744 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:59.697745 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:59.697746 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:37:59.697750 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:59.697767 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:37:59.697771 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:59.697848 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:59.697940 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 16:37:59.697966 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 16:37:59.698160 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.194:0
--local_ip_for_outbound_sockets=127.0.253.194
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:45951
--builtin_ntp_servers=127.0.253.212:39713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 16:37:59.698292 18221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:59.693369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:45951 (local address 127.0.253.193:37815)
0812 16:37:59.693652 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:59.693688 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:59.694194 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:59.694398 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:59.694401 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:59.694409 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:59.694918 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:59.694922 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:59.695661 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:59.695661 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:59.696437 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:59.696439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:59.696794 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:59.696795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:59.696958 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:59.697605 (+ 647us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:37:59.697605 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:59.697635 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:37:59.697657 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:59.697852 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:59.697859 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:59.697863 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:37:59.697942 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 16:37:59.698052 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":117,"threads_started":1}
I20260812 16:37:59.698782 18217 heartbeater.cc:347] Connected to a master server at 127.0.253.254:45951
I20260812 16:37:59.698894 18217 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:59.699115 18217 heartbeater.cc:511] Master 127.0.253.254:45951 requested a full tablet report, sending...
I20260812 16:37:59.699705 17895 ts_manager.cc:194] Registered new tserver with Master: 6445f670cea147a586a4b1e4cbcb34f2 (127.0.253.193:41657)
I20260812 16:37:59.700659 17895 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.253.193:37815
W20260812 16:37:59.785082 18222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:59.785284 18222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:59.785307 18222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:59.786399 18222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:59.786433 18222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:59.786464 18222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.194
I20260812 16:37:59.787787 18222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:39713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:45951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.194
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:59.787989 18222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:59.788255 18222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:59.793593 18231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:59.793587 18232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:59.793645 18235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:59.793612 18222 server_base.cc:1034] running on GCE node
I20260812 16:37:59.794000 18222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:59.794257 18222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:59.795439 18222 hybrid_clock.cc:648] HybridClock initialized: now 1786552679795422 us; error 26 us; skew 500 ppm
I20260812 16:37:59.796820 18222 webserver.cc:460] Webserver started at http://127.0.253.194:41489/ using document root <none> and password file <none>
I20260812 16:37:59.797122 18222 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:59.797202 18222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:59.797369 18222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:59.798509 18222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/data/instance:
uuid: "44625a25ea4347a0a9d557e95d5b2ff5"
format_stamp: "Formatted at 2026-08-12 16:37:59 on dist-test-slave-w4v5"
server_key: "9e63467c17e387c106aa063b46407e35"
server_key_iv: "b2b6faead9af55ec0f717839203dd6e4"
server_key_version: "encryptionkey@0"
I20260812 16:37:59.798900 18222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance:
uuid: "44625a25ea4347a0a9d557e95d5b2ff5"
format_stamp: "Formatted at 2026-08-12 16:37:59 on dist-test-slave-w4v5"
server_key: "9e63467c17e387c106aa063b46407e35"
server_key_iv: "b2b6faead9af55ec0f717839203dd6e4"
server_key_version: "encryptionkey@0"
I20260812 16:37:59.800459 18222 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:37:59.801368 18246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:59.801543 18222 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:37:59.801622 18222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/wal
uuid: "44625a25ea4347a0a9d557e95d5b2ff5"
format_stamp: "Formatted at 2026-08-12 16:37:59 on dist-test-slave-w4v5"
server_key: "9e63467c17e387c106aa063b46407e35"
server_key_iv: "b2b6faead9af55ec0f717839203dd6e4"
server_key_version: "encryptionkey@0"
I20260812 16:37:59.801676 18222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:59.816020 18222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:59.816305 18222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:59.816424 18222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:59.816612 18222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:37:59.816993 18222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:59.817032 18222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:59.817092 18222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:59.817124 18222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:59.823122 18222 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.194:34001
I20260812 16:37:59.823189 18449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.194:34001 every 8 connection(s)
I20260812 16:37:59.823578 18222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
I20260812 16:37:59.824867 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 18222
I20260812 16:37:59.824942 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance
I20260812 16:37:59.825132 1015 external_mini_cluster.cc:1428] Setting key b4496c563dc9adeb2c802c116c6a541f
I20260812 16:37:59.826093 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.195:0
--local_ip_for_outbound_sockets=127.0.253.195
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:45951
--builtin_ntp_servers=127.0.253.212:39713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 16:37:59.829378 17969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:59.824415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:41577 (local address 127.0.253.254:45951)
0812 16:37:59.824486 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:59.824488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:59.824728 (+ 240us) server_negotiation.cc:407] Connection header received
0812 16:37:59.825365 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:59.825366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:59.825379 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:59.825440 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:59.826130 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:59.826700 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:59.827815 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:59.828039 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:59.828122 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:59.829069 (+ 947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:59.829070 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:59.829070 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:37:59.829076 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:59.829092 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:37:59.829097 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:59.829150 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:59.829262 (+ 112us) server_negotiation.cc:299] Negotiation successful
0812 16:37:59.829294 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 16:37:59.829677 18453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:59.824400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:45951 (local address 127.0.253.194:41577)
0812 16:37:59.824590 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:59.824631 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:59.825246 (+ 615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:59.825510 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:59.825514 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:59.825524 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:59.826031 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:59.826036 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:59.826812 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:59.826813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:59.827712 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:59.827715 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:59.828143 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:59.828144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:59.828291 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:59.828899 (+ 608us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:37:59.828899 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:59.828930 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:37:59.828955 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:59.829162 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:59.829168 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:59.829171 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:37:59.829264 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 16:37:59.829397 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":86,"threads_started":1}
I20260812 16:37:59.830251 18451 heartbeater.cc:347] Connected to a master server at 127.0.253.254:45951
I20260812 16:37:59.830385 18451 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:59.830636 18451 heartbeater.cc:511] Master 127.0.253.254:45951 requested a full tablet report, sending...
I20260812 16:37:59.831126 17895 ts_manager.cc:194] Registered new tserver with Master: 44625a25ea4347a0a9d557e95d5b2ff5 (127.0.253.194:34001)
I20260812 16:37:59.831684 17895 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.253.194:41577
W20260812 16:37:59.912118 18454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:37:59.912338 18454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:37:59.912360 18454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:37:59.913455 18454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:37:59.913486 18454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:37:59.913511 18454 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.195
I20260812 16:37:59.914786 18454 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:39713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:45951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.195
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:37:59.914976 18454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:37:59.915262 18454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:37:59.920470 18464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:59.920436 18465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:37:59.920454 18467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:37:59.920771 18454 server_base.cc:1034] running on GCE node
I20260812 16:37:59.921016 18454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:37:59.921270 18454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:37:59.922475 18454 hybrid_clock.cc:648] HybridClock initialized: now 1786552679922435 us; error 57 us; skew 500 ppm
I20260812 16:37:59.923967 18454 webserver.cc:460] Webserver started at http://127.0.253.195:43305/ using document root <none> and password file <none>
I20260812 16:37:59.924263 18454 fs_manager.cc:359] Metadata directory not provided
I20260812 16:37:59.924338 18454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:37:59.924500 18454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:37:59.925639 18454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/data/instance:
uuid: "117b79e2dc494993a98a6af16125afba"
format_stamp: "Formatted at 2026-08-12 16:37:59 on dist-test-slave-w4v5"
server_key: "dff0f90c2942044d9ce722618180b620"
server_key_iv: "94e25388f288e6035023a5a050f43cf7"
server_key_version: "encryptionkey@0"
I20260812 16:37:59.926038 18454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance:
uuid: "117b79e2dc494993a98a6af16125afba"
format_stamp: "Formatted at 2026-08-12 16:37:59 on dist-test-slave-w4v5"
server_key: "dff0f90c2942044d9ce722618180b620"
server_key_iv: "94e25388f288e6035023a5a050f43cf7"
server_key_version: "encryptionkey@0"
I20260812 16:37:59.927582 18454 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260812 16:37:59.928401 18478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:59.928642 18454 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:37:59.928812 18454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/wal
uuid: "117b79e2dc494993a98a6af16125afba"
format_stamp: "Formatted at 2026-08-12 16:37:59 on dist-test-slave-w4v5"
server_key: "dff0f90c2942044d9ce722618180b620"
server_key_iv: "94e25388f288e6035023a5a050f43cf7"
server_key_version: "encryptionkey@0"
I20260812 16:37:59.928905 18454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:37:59.947801 18454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:37:59.948117 18454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:37:59.948274 18454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:37:59.948515 18454 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:37:59.948864 18454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:37:59.948920 18454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:59.948962 18454 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:37:59.948990 18454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:37:59.954712 18454 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.195:34743
I20260812 16:37:59.954798 18703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.195:34743 every 8 connection(s)
I20260812 16:37:59.955163 18454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
I20260812 16:37:59.960438 17969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:59.956042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:35733 (local address 127.0.253.254:45951)
0812 16:37:59.956099 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:59.956102 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:59.956319 (+ 217us) server_negotiation.cc:407] Connection header received
0812 16:37:59.956835 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:59.956837 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:59.956849 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:59.956924 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:59.957612 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:59.958102 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:59.959053 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:37:59.959326 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:37:59.959392 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:59.960191 (+ 799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:37:59.960192 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:37:59.960193 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:37:59.960197 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:37:59.960213 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:37:59.960217 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:37:59.960268 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 16:37:59.960350 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 16:37:59.960374 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 16:37:59.960670 18711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:37:59.955951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:45951 (local address 127.0.253.195:35733)
0812 16:37:59.956165 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 16:37:59.956209 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 16:37:59.956704 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:37:59.956999 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:37:59.957002 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:37:59.957011 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:37:59.957529 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:59.957533 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:59.958181 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:59.958181 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:59.958954 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:37:59.958956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:37:59.959411 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:37:59.959411 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:37:59.959527 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:37:59.960051 (+ 524us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:37:59.960051 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:37:59.960081 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:37:59.960103 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:37:59.960277 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:37:59.960281 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:37:59.960283 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 16:37:59.960337 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 16:37:59.960419 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":86,"threads_started":1}
I20260812 16:37:59.961345 18708 heartbeater.cc:347] Connected to a master server at 127.0.253.254:45951
I20260812 16:37:59.961427 18708 heartbeater.cc:464] Registering TS with master...
I20260812 16:37:59.961616 18708 heartbeater.cc:511] Master 127.0.253.254:45951 requested a full tablet report, sending...
I20260812 16:37:59.962059 17895 ts_manager.cc:194] Registered new tserver with Master: 117b79e2dc494993a98a6af16125afba (127.0.253.195:34743)
I20260812 16:37:59.962474 17895 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.253.195:35733
I20260812 16:37:59.962774 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 18454
I20260812 16:37:59.962874 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance
I20260812 16:37:59.963099 1015 external_mini_cluster.cc:1428] Setting key f5dad32603682e67b6cd084babaa9c0a
I20260812 16:37:59.963985 1015 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 16:37:59.965893 17969 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:37:59.965361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43896 (local address 127.0.253.254:45951)
0812 16:37:59.965401 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 16:37:59.965403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:37:59.965566 (+ 163us) server_negotiation.cc:407] Connection header received
0812 16:37:59.965596 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:37:59.965597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:37:59.965606 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:37:59.965661 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:37:59.965809 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43896: BlockingRecv error: recv got EOF from 127.0.0.1:43896 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 16:37:59.965917 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 18000
I20260812 16:37:59.984547 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 18222
I20260812 16:38:00.005605 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 18454
I20260812 16:38:00.020653 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 17848
2026-08-12T16:38:00Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (664 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-12T16:38:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:38:00Z Disabled control of system clock
I20260812 16:38:00.057744 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:46225
--webserver_interface=127.0.253.254
--webserver_port=0
--builtin_ntp_servers=127.0.253.212:41965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:46225
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 16:38:00.141986 18734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:38:00.142280 18734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:38:00.142302 18734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:38:00.143472 18734 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:38:00.143517 18734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:38:00.143532 18734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:38:00.143577 18734 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:38:00.143610 18734 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:38:00.144956 18734 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:41965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:46225
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:46225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:38:00.145224 18734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:38:00.145552 18734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:38:00.150470 18746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:38:00.150547 18743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:38:00.150586 18748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:38:00.150848 18734 server_base.cc:1034] running on GCE node
I20260812 16:38:00.151163 18734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:38:00.151448 18734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:38:00.152628 18734 hybrid_clock.cc:648] HybridClock initialized: now 1786552680152617 us; error 40 us; skew 500 ppm
I20260812 16:38:00.153945 18734 webserver.cc:460] Webserver started at http://127.0.253.254:42219/ using document root <none> and password file <none>
I20260812 16:38:00.154248 18734 fs_manager.cc:359] Metadata directory not provided
I20260812 16:38:00.154325 18734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:38:00.154490 18734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:38:00.155673 18734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "2d4b3d54f0b741c59d7955129942b68b"
format_stamp: "Formatted at 2026-08-12 16:38:00 on dist-test-slave-w4v5"
server_key: "d49fec809381665412d1ae904f4e3779"
server_key_iv: "a18f3a4679eaffed1046f9d1e99931e2"
server_key_version: "encryptionkey@0"
I20260812 16:38:00.156049 18734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "2d4b3d54f0b741c59d7955129942b68b"
format_stamp: "Formatted at 2026-08-12 16:38:00 on dist-test-slave-w4v5"
server_key: "d49fec809381665412d1ae904f4e3779"
server_key_iv: "a18f3a4679eaffed1046f9d1e99931e2"
server_key_version: "encryptionkey@0"
I20260812 16:38:00.157379 18734 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:38:00.158223 18760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:38:00.158428 18734 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:38:00.158545 18734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "2d4b3d54f0b741c59d7955129942b68b"
format_stamp: "Formatted at 2026-08-12 16:38:00 on dist-test-slave-w4v5"
server_key: "d49fec809381665412d1ae904f4e3779"
server_key_iv: "a18f3a4679eaffed1046f9d1e99931e2"
server_key_version: "encryptionkey@0"
I20260812 16:38:00.158624 18734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:38:00.162238 18734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:38:00.162592 18734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:38:00.162738 18734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:38:00.167065 18734 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:46225
I20260812 16:38:00.167163 18845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:46225 every 8 connection(s)
I20260812 16:38:00.167519 18734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:38:00.168256 18850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:38:00.171223 18850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d4b3d54f0b741c59d7955129942b68b: Bootstrap starting.
I20260812 16:38:00.171844 18850 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d4b3d54f0b741c59d7955129942b68b: Neither blocks nor log segments found. Creating new log.
I20260812 16:38:00.172169 18850 log.cc:824] T 00000000000000000000000000000000 P 2d4b3d54f0b741c59d7955129942b68b: Log is configured to *not* fsync() on all Append() calls
I20260812 16:38:00.172648 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 18734
I20260812 16:38:00.172757 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:38:00.172959 1015 external_mini_cluster.cc:1428] Setting key feb5c6aab9ab4c7e38fb84ba65641d53
I20260812 16:38:00.173084 18850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d4b3d54f0b741c59d7955129942b68b: No bootstrap required, opened a new log
I20260812 16:38:00.175025 18850 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d4b3d54f0b741c59d7955129942b68b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d4b3d54f0b741c59d7955129942b68b" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 46225 } }
I20260812 16:38:00.175293 18850 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d4b3d54f0b741c59d7955129942b68b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:38:00.175349 18850 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d4b3d54f0b741c59d7955129942b68b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d4b3d54f0b741c59d7955129942b68b, State: Initialized, Role: FOLLOWER
I20260812 16:38:00.175505 18850 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d4b3d54f0b741c59d7955129942b68b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d4b3d54f0b741c59d7955129942b68b" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 46225 } }
I20260812 16:38:00.175618 18850 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d4b3d54f0b741c59d7955129942b68b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:38:00.175664 18850 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d4b3d54f0b741c59d7955129942b68b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:38:00.175722 18850 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d4b3d54f0b741c59d7955129942b68b [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:38:00.176471 18850 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d4b3d54f0b741c59d7955129942b68b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d4b3d54f0b741c59d7955129942b68b" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 46225 } }
I20260812 16:38:00.176496 18856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:00.173539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34078 (local address 127.0.253.254:46225)
0812 16:38:00.173787 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0812 16:38:00.173794 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:38:00.173819 (+ 25us) server_negotiation.cc:407] Connection header received
0812 16:38:00.175015 (+ 1196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:38:00.175024 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:38:00.175107 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:38:00.175245 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:38:00.176011 (+ 766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:38:00.176016 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:38:00.176016 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:38:00.176021 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:38:00.176065 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:38:00.176077 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:38:00.176133 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 16:38:00.176224 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 16:38:00.176296 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":119,"threads_started":1}
I20260812 16:38:00.176565 18850 leader_election.cc:302] T 00000000000000000000000000000000 P 2d4b3d54f0b741c59d7955129942b68b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d4b3d54f0b741c59d7955129942b68b; no voters:
I20260812 16:38:00.176750 18850 leader_election.cc:288] T 00000000000000000000000000000000 P 2d4b3d54f0b741c59d7955129942b68b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:38:00.177011 18850 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d4b3d54f0b741c59d7955129942b68b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:38:00.177062 18858 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d4b3d54f0b741c59d7955129942b68b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:38:00.177202 18858 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d4b3d54f0b741c59d7955129942b68b [term 1 LEADER]: Becoming Leader. State: Replica: 2d4b3d54f0b741c59d7955129942b68b, State: Running, Role: LEADER
I20260812 16:38:00.177299 18858 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d4b3d54f0b741c59d7955129942b68b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d4b3d54f0b741c59d7955129942b68b" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 46225 } }
I20260812 16:38:00.177688 18862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d4b3d54f0b741c59d7955129942b68b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d4b3d54f0b741c59d7955129942b68b. Latest consensus state: current_term: 1 leader_uuid: "2d4b3d54f0b741c59d7955129942b68b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d4b3d54f0b741c59d7955129942b68b" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 46225 } } }
I20260812 16:38:00.177774 18862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d4b3d54f0b741c59d7955129942b68b [sys.catalog]: This master's current role is: LEADER
I20260812 16:38:00.177935 18860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d4b3d54f0b741c59d7955129942b68b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d4b3d54f0b741c59d7955129942b68b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d4b3d54f0b741c59d7955129942b68b" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 46225 } } }
I20260812 16:38:00.177989 18872 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:38:00.178634 18872 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:38:00.178011 18860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d4b3d54f0b741c59d7955129942b68b [sys.catalog]: This master's current role is: LEADER
I20260812 16:38:00.180498 18872 catalog_manager.cc:1347] Generated new cluster ID: 2f396d88f44b4702a96bf53c12168873
I20260812 16:38:00.180589 18872 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:38:00.189963 18872 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:38:00.190158 18872 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:38:00.193850 18872 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d4b3d54f0b741c59d7955129942b68b: Generated new TSK 0
I20260812 16:38:00.194052 18872 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:38:00.202476 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.193:0
--local_ip_for_outbound_sockets=127.0.253.193
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:46225
--builtin_ntp_servers=127.0.253.212:41965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 16:38:00.304742 18884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:38:00.304957 18884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:38:00.304979 18884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:38:00.306144 18884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:38:00.306185 18884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:38:00.306212 18884 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.193
I20260812 16:38:00.307651 18884 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:41965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:46225
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.193
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:38:00.307912 18884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:38:00.308210 18884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:38:00.313836 18895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:38:00.313964 18901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:38:00.313840 18896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:38:00.314698 18884 server_base.cc:1034] running on GCE node
I20260812 16:38:00.314929 18884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:38:00.315243 18884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:38:00.316432 18884 hybrid_clock.cc:648] HybridClock initialized: now 1786552680316406 us; error 40 us; skew 500 ppm
I20260812 16:38:00.317889 18884 webserver.cc:460] Webserver started at http://127.0.253.193:33583/ using document root <none> and password file <none>
I20260812 16:38:00.318176 18884 fs_manager.cc:359] Metadata directory not provided
I20260812 16:38:00.318223 18884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:38:00.318400 18884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:38:00.319605 18884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/data/instance:
uuid: "96d26ccd53d74cc88a639615ad0c44ba"
format_stamp: "Formatted at 2026-08-12 16:38:00 on dist-test-slave-w4v5"
server_key: "8d74b473d3e38bf94b36420ee28e7ed1"
server_key_iv: "f5fdd93bd2ea1b1d27fa642c0225bfa0"
server_key_version: "encryptionkey@0"
I20260812 16:38:00.319986 18884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance:
uuid: "96d26ccd53d74cc88a639615ad0c44ba"
format_stamp: "Formatted at 2026-08-12 16:38:00 on dist-test-slave-w4v5"
server_key: "8d74b473d3e38bf94b36420ee28e7ed1"
server_key_iv: "f5fdd93bd2ea1b1d27fa642c0225bfa0"
server_key_version: "encryptionkey@0"
I20260812 16:38:00.321403 18884 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:38:00.322238 18909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:38:00.322454 18884 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:38:00.322556 18884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/wal
uuid: "96d26ccd53d74cc88a639615ad0c44ba"
format_stamp: "Formatted at 2026-08-12 16:38:00 on dist-test-slave-w4v5"
server_key: "8d74b473d3e38bf94b36420ee28e7ed1"
server_key_iv: "f5fdd93bd2ea1b1d27fa642c0225bfa0"
server_key_version: "encryptionkey@0"
I20260812 16:38:00.322650 18884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:38:00.326210 18884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:38:00.326545 18884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:38:00.326704 18884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:38:00.326933 18884 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:38:00.327328 18884 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:38:00.327389 18884 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:38:00.327432 18884 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:38:00.327461 18884 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:38:00.333143 18884 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.193:36363
I20260812 16:38:00.333247 19119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.193:36363 every 8 connection(s)
I20260812 16:38:00.333582 18884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
I20260812 16:38:00.336822 18856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:00.334492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:48611 (local address 127.0.253.254:46225)
0812 16:38:00.334593 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 16:38:00.334595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:38:00.334782 (+ 187us) server_negotiation.cc:407] Connection header received
0812 16:38:00.335347 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:38:00.335349 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:38:00.335355 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:38:00.335420 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:38:00.336595 (+ 1175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:38:00.336596 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:38:00.336596 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:38:00.336600 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:38:00.336615 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:38:00.336619 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:38:00.336670 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 16:38:00.336758 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 16:38:00.336771 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 16:38:00.337028 19123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:00.334436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:46225 (local address 127.0.253.193:48611)
0812 16:38:00.334635 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 16:38:00.334676 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 16:38:00.335246 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:38:00.335502 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:38:00.335505 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:38:00.335515 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:38:00.336440 (+ 925us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:38:00.336441 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:38:00.336470 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:38:00.336499 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:38:00.336684 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:38:00.336689 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:38:00.336691 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 16:38:00.336757 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 16:38:00.336809 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":99,"threads_started":1}
I20260812 16:38:00.337594 19120 heartbeater.cc:347] Connected to a master server at 127.0.253.254:46225
I20260812 16:38:00.337721 19120 heartbeater.cc:464] Registering TS with master...
I20260812 16:38:00.337905 19120 heartbeater.cc:511] Master 127.0.253.254:46225 requested a full tablet report, sending...
I20260812 16:38:00.338392 18788 ts_manager.cc:194] Registered new tserver with Master: 96d26ccd53d74cc88a639615ad0c44ba (127.0.253.193:36363)
I20260812 16:38:00.339820 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 18884
I20260812 16:38:00.339903 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance
I20260812 16:38:00.340121 1015 external_mini_cluster.cc:1428] Setting key a75e9e59f9c9a1d3611c6824c8a454fb
I20260812 16:38:00.341051 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.194:0
--local_ip_for_outbound_sockets=127.0.253.194
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:46225
--builtin_ntp_servers=127.0.253.212:41965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 16:38:00.426487 19124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:38:00.426703 19124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:38:00.426733 19124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:38:00.427955 19124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:38:00.427997 19124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:38:00.428023 19124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.194
I20260812 16:38:00.429415 19124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:41965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:46225
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.194
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:38:00.429680 19124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:38:00.429982 19124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:38:00.435195 19135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:38:00.435271 19138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:38:00.435395 19132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:38:00.435384 19124 server_base.cc:1034] running on GCE node
I20260812 16:38:00.435681 19124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:38:00.435917 19124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:38:00.437067 19124 hybrid_clock.cc:648] HybridClock initialized: now 1786552680437042 us; error 41 us; skew 500 ppm
I20260812 16:38:00.438297 19124 webserver.cc:460] Webserver started at http://127.0.253.194:38309/ using document root <none> and password file <none>
I20260812 16:38:00.438491 19124 fs_manager.cc:359] Metadata directory not provided
I20260812 16:38:00.438532 19124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:38:00.438622 19124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:38:00.439775 19124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/data/instance:
uuid: "36c5d00f584f491d83c2e6d858d46fbe"
format_stamp: "Formatted at 2026-08-12 16:38:00 on dist-test-slave-w4v5"
server_key: "db50f8c31a9a0549c6478feee8c7c19e"
server_key_iv: "0ede8c9298fe6c4c525de114ada70499"
server_key_version: "encryptionkey@0"
I20260812 16:38:00.440095 19124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance:
uuid: "36c5d00f584f491d83c2e6d858d46fbe"
format_stamp: "Formatted at 2026-08-12 16:38:00 on dist-test-slave-w4v5"
server_key: "db50f8c31a9a0549c6478feee8c7c19e"
server_key_iv: "0ede8c9298fe6c4c525de114ada70499"
server_key_version: "encryptionkey@0"
I20260812 16:38:00.441437 19124 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:38:00.442198 19151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:38:00.442397 19124 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:38:00.442487 19124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/wal
uuid: "36c5d00f584f491d83c2e6d858d46fbe"
format_stamp: "Formatted at 2026-08-12 16:38:00 on dist-test-slave-w4v5"
server_key: "db50f8c31a9a0549c6478feee8c7c19e"
server_key_iv: "0ede8c9298fe6c4c525de114ada70499"
server_key_version: "encryptionkey@0"
I20260812 16:38:00.442538 19124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:38:00.445924 19124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:38:00.446161 19124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:38:00.446271 19124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:38:00.446439 19124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:38:00.446735 19124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:38:00.446767 19124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:38:00.446789 19124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:38:00.446800 19124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:38:00.452561 19124 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.194:39277
I20260812 16:38:00.452670 19356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.194:39277 every 8 connection(s)
I20260812 16:38:00.453004 19124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
I20260812 16:38:00.456135 18856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:00.453812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:51193 (local address 127.0.253.254:46225)
0812 16:38:00.453876 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 16:38:00.453879 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:38:00.454097 (+ 218us) server_negotiation.cc:407] Connection header received
0812 16:38:00.454552 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:38:00.454553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:38:00.454558 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:38:00.454622 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:38:00.455940 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:38:00.455941 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:38:00.455941 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:38:00.455945 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:38:00.455963 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:38:00.455967 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:38:00.456019 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 16:38:00.456068 (+ 49us) server_negotiation.cc:299] Negotiation successful
0812 16:38:00.456080 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 16:38:00.456323 19361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:00.453759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:46225 (local address 127.0.253.194:51193)
0812 16:38:00.453965 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 16:38:00.454006 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 16:38:00.454467 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:38:00.454687 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:38:00.454690 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:38:00.454699 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:38:00.455789 (+ 1090us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:38:00.455789 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:38:00.455820 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:38:00.455850 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:38:00.456015 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:38:00.456017 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:38:00.456020 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:38:00.456067 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 16:38:00.456116 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":105,"threads_started":1}
I20260812 16:38:00.456862 19357 heartbeater.cc:347] Connected to a master server at 127.0.253.254:46225
I20260812 16:38:00.456902 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 19124
I20260812 16:38:00.456959 19357 heartbeater.cc:464] Registering TS with master...
I20260812 16:38:00.457006 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance
I20260812 16:38:00.457129 19357 heartbeater.cc:511] Master 127.0.253.254:46225 requested a full tablet report, sending...
I20260812 16:38:00.457247 1015 external_mini_cluster.cc:1428] Setting key f17ad2e930b02f63ec6da5c4c2edebb4
I20260812 16:38:00.457607 18788 ts_manager.cc:194] Registered new tserver with Master: 36c5d00f584f491d83c2e6d858d46fbe (127.0.253.194:39277)
I20260812 16:38:00.458405 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.195:0
--local_ip_for_outbound_sockets=127.0.253.195
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:46225
--builtin_ntp_servers=127.0.253.212:41965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 16:38:00.544565 19362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:38:00.544773 19362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:38:00.544795 19362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:38:00.546051 19362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:38:00.546094 19362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:38:00.546154 19362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.195
I20260812 16:38:00.547622 19362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:41965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:46225
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.195
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:38:00.547874 19362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:38:00.548177 19362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:38:00.553483 19385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:38:00.553512 19383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:38:00.553632 19379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:38:00.553798 19362 server_base.cc:1034] running on GCE node
I20260812 16:38:00.554078 19362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:38:00.554296 19362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:38:00.555459 19362 hybrid_clock.cc:648] HybridClock initialized: now 1786552680555456 us; error 43 us; skew 500 ppm
I20260812 16:38:00.556725 19362 webserver.cc:460] Webserver started at http://127.0.253.195:39467/ using document root <none> and password file <none>
I20260812 16:38:00.556978 19362 fs_manager.cc:359] Metadata directory not provided
I20260812 16:38:00.557080 19362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:38:00.557225 19362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:38:00.558290 19362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/data/instance:
uuid: "bc229f7be3a34bc3ba959b34b76eb39b"
format_stamp: "Formatted at 2026-08-12 16:38:00 on dist-test-slave-w4v5"
server_key: "e2d563b0853c2ffed79e43222b18a1f7"
server_key_iv: "9cc99a68c09f034f41c5e1a6c1eec968"
server_key_version: "encryptionkey@0"
I20260812 16:38:00.558663 19362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance:
uuid: "bc229f7be3a34bc3ba959b34b76eb39b"
format_stamp: "Formatted at 2026-08-12 16:38:00 on dist-test-slave-w4v5"
server_key: "e2d563b0853c2ffed79e43222b18a1f7"
server_key_iv: "9cc99a68c09f034f41c5e1a6c1eec968"
server_key_version: "encryptionkey@0"
I20260812 16:38:00.560163 19362 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:38:00.560982 19395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:38:00.561236 19362 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:38:00.561386 19362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/wal
uuid: "bc229f7be3a34bc3ba959b34b76eb39b"
format_stamp: "Formatted at 2026-08-12 16:38:00 on dist-test-slave-w4v5"
server_key: "e2d563b0853c2ffed79e43222b18a1f7"
server_key_iv: "9cc99a68c09f034f41c5e1a6c1eec968"
server_key_version: "encryptionkey@0"
I20260812 16:38:00.561479 19362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:38:00.565352 19362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:38:00.565637 19362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:38:00.565785 19362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:38:00.566008 19362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:38:00.566344 19362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:38:00.566401 19362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:38:00.566442 19362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:38:00.566471 19362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:38:00.572491 19362 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.195:43741
I20260812 16:38:00.572546 19599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.195:43741 every 8 connection(s)
I20260812 16:38:00.572901 19362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
I20260812 16:38:00.574352 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 19362
I20260812 16:38:00.574432 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance
I20260812 16:38:00.574637 1015 external_mini_cluster.cc:1428] Setting key c8ff499aaf1605d4fdb4690801328bdd
I20260812 16:38:00.576732 18856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:00.573744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:47397 (local address 127.0.253.254:46225)
0812 16:38:00.573789 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 16:38:00.573791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:38:00.574024 (+ 233us) server_negotiation.cc:407] Connection header received
0812 16:38:00.574923 (+ 899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:38:00.574925 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:38:00.574936 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:38:00.574997 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:38:00.576462 (+ 1465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:38:00.576464 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:38:00.576464 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:38:00.576469 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:38:00.576484 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:38:00.576489 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:38:00.576560 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 16:38:00.576644 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 16:38:00.576664 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 16:38:00.576967 19606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:00.573719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:46225 (local address 127.0.253.195:47397)
0812 16:38:00.573897 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 16:38:00.573930 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 16:38:00.574605 (+ 675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:38:00.575094 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:38:00.575098 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:38:00.575109 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:38:00.576258 (+ 1149us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:38:00.576259 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:38:00.576288 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:38:00.576371 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:38:00.576558 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:38:00.576564 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:38:00.576568 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:38:00.576650 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 16:38:00.576713 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":71,"threads_started":1}
I20260812 16:38:00.577512 19600 heartbeater.cc:347] Connected to a master server at 127.0.253.254:46225
I20260812 16:38:00.577615 19600 heartbeater.cc:464] Registering TS with master...
I20260812 16:38:00.577769 19600 heartbeater.cc:511] Master 127.0.253.254:46225 requested a full tablet report, sending...
I20260812 16:38:00.578151 18786 ts_manager.cc:194] Registered new tserver with Master: bc229f7be3a34bc3ba959b34b76eb39b (127.0.253.195:43741)
I20260812 16:38:00.586763 1015 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 16:38:00.588737 18856 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:38:00.588215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34092 (local address 127.0.253.254:46225)
0812 16:38:00.588279 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 16:38:00.588282 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:38:00.588371 (+ 89us) server_negotiation.cc:407] Connection header received
0812 16:38:00.588461 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:38:00.588462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:38:00.588468 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:38:00.588514 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:38:00.588660 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34092: BlockingRecv error: recv got EOF from 127.0.0.1:34092 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 16:38:00.588841 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 18884
I20260812 16:38:00.606918 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 19124
I20260812 16:38:00.624893 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 19362
I20260812 16:38:00.633374 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 18734
2026-08-12T16:38:00Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (612 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:38:00 dist-test-slave-w4v5 krb5kdc[19621](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:38:00 dist-test-slave-w4v5 krb5kdc[19621](info): set up 2 sockets
Aug 12 16:38:00 dist-test-slave-w4v5 krb5kdc[19621](info): commencing operation
krb5kdc: starting...
W20260812 16:38:01.952013 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.291s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:38:02 dist-test-slave-w4v5 krb5kdc[19621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:38:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:38:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
I20260812 16:38:02.087049 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:38127
--webserver_interface=127.0.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:34419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:38127
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:38:02.171450 19641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:38:02.171686 19641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:38:02.171712 19641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:38:02.172914 19641 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:38:02.172956 19641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:38:02.172971 19641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:38:02.172991 19641 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:38:02.173003 19641 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:38:02.174470 19641 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:34419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:38127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:38127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:38:02.174679 19641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:38:02.174856 19641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:38:02.179939 19650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:38:02.179994 19653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:38:02.179982 19651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:38:02.180132 19641 server_base.cc:1034] running on GCE node
I20260812 16:38:02.180509 19641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:38:02.180828 19641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:38:02.181989 19641 hybrid_clock.cc:648] HybridClock initialized: now 1786552682181972 us; error 43 us; skew 500 ppm
Aug 12 16:38:02 dist-test-slave-w4v5 krb5kdc[19621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552682, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:38:02.185005 19641 init.cc:377] Logged in from keytab as kudu/127.0.253.254@KRBTEST.COM (short username kudu)
I20260812 16:38:02.185545 19641 webserver.cc:460] Webserver started at http://127.0.253.254:33039/ using document root <none> and password file <none>
I20260812 16:38:02.185814 19641 fs_manager.cc:359] Metadata directory not provided
I20260812 16:38:02.185863 19641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:38:02.186015 19641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:38:02.187202 19641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "43479def7a5140d9aaaee178f204f7bc"
format_stamp: "Formatted at 2026-08-12 16:38:02 on dist-test-slave-w4v5"
server_key: "17631445f7038bfd54955f9f808d7a60"
server_key_iv: "03e6c0137659858e42eb5e4b3920abf3"
server_key_version: "encryptionkey@0"
I20260812 16:38:02.187592 19641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "43479def7a5140d9aaaee178f204f7bc"
format_stamp: "Formatted at 2026-08-12 16:38:02 on dist-test-slave-w4v5"
server_key: "17631445f7038bfd54955f9f808d7a60"
server_key_iv: "03e6c0137659858e42eb5e4b3920abf3"
server_key_version: "encryptionkey@0"
I20260812 16:38:02.188984 19641 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:38:02.189824 19665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:38:02.190119 19641 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:38:02.190219 19641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "43479def7a5140d9aaaee178f204f7bc"
format_stamp: "Formatted at 2026-08-12 16:38:02 on dist-test-slave-w4v5"
server_key: "17631445f7038bfd54955f9f808d7a60"
server_key_iv: "03e6c0137659858e42eb5e4b3920abf3"
server_key_version: "encryptionkey@0"
I20260812 16:38:02.190296 19641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:38:02.204819 19641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:38:02.207309 19641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:38:02.207479 19641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:38:02.212231 19641 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:38127
I20260812 16:38:02.212312 19745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:38127 every 8 connection(s)
I20260812 16:38:02.212659 19641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:38:02.213402 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 19641
I20260812 16:38:02.213493 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:38:02.213726 1015 external_mini_cluster.cc:1428] Setting key 3d493e6fdd29a1d77ebf75b5aaa7504a
I20260812 16:38:02.213896 19749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:38:02.217542 19749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43479def7a5140d9aaaee178f204f7bc: Bootstrap starting.
I20260812 16:38:02.218354 19749 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43479def7a5140d9aaaee178f204f7bc: Neither blocks nor log segments found. Creating new log.
I20260812 16:38:02.218698 19749 log.cc:824] T 00000000000000000000000000000000 P 43479def7a5140d9aaaee178f204f7bc: Log is configured to *not* fsync() on all Append() calls
I20260812 16:38:02.219581 19749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43479def7a5140d9aaaee178f204f7bc: No bootstrap required, opened a new log
Aug 12 16:38:02 dist-test-slave-w4v5 krb5kdc[19621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:38:02.221767 19749 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43479def7a5140d9aaaee178f204f7bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43479def7a5140d9aaaee178f204f7bc" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 38127 } }
I20260812 16:38:02.221908 19749 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43479def7a5140d9aaaee178f204f7bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:38:02.221932 19749 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43479def7a5140d9aaaee178f204f7bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43479def7a5140d9aaaee178f204f7bc, State: Initialized, Role: FOLLOWER
I20260812 16:38:02.222097 19749 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43479def7a5140d9aaaee178f204f7bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43479def7a5140d9aaaee178f204f7bc" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 38127 } }
I20260812 16:38:02.222211 19749 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43479def7a5140d9aaaee178f204f7bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:38:02.222252 19749 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43479def7a5140d9aaaee178f204f7bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:38:02.222303 19749 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43479def7a5140d9aaaee178f204f7bc [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:38:02.222940 19749 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43479def7a5140d9aaaee178f204f7bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43479def7a5140d9aaaee178f204f7bc" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 38127 } }
I20260812 16:38:02.223058 19749 leader_election.cc:302] T 00000000000000000000000000000000 P 43479def7a5140d9aaaee178f204f7bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43479def7a5140d9aaaee178f204f7bc; no voters:
I20260812 16:38:02.223311 19749 leader_election.cc:288] T 00000000000000000000000000000000 P 43479def7a5140d9aaaee178f204f7bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:38:02.223340 19758 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43479def7a5140d9aaaee178f204f7bc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:38:02.223459 19758 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43479def7a5140d9aaaee178f204f7bc [term 1 LEADER]: Becoming Leader. State: Replica: 43479def7a5140d9aaaee178f204f7bc, State: Running, Role: LEADER
I20260812 16:38:02.223610 19749 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43479def7a5140d9aaaee178f204f7bc [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:38:02.223623 19758 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43479def7a5140d9aaaee178f204f7bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43479def7a5140d9aaaee178f204f7bc" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 38127 } }
I20260812 16:38:02.224125 19760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43479def7a5140d9aaaee178f204f7bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43479def7a5140d9aaaee178f204f7bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43479def7a5140d9aaaee178f204f7bc" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 38127 } } }
I20260812 16:38:02.224397 19760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43479def7a5140d9aaaee178f204f7bc [sys.catalog]: This master's current role is: LEADER
I20260812 16:38:02.224320 19759 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43479def7a5140d9aaaee178f204f7bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43479def7a5140d9aaaee178f204f7bc. Latest consensus state: current_term: 1 leader_uuid: "43479def7a5140d9aaaee178f204f7bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43479def7a5140d9aaaee178f204f7bc" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 38127 } } }
I20260812 16:38:02.224866 19759 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43479def7a5140d9aaaee178f204f7bc [sys.catalog]: This master's current role is: LEADER
I20260812 16:38:02.225093 19774 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:38:02.225695 19774 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:38:02.226536 19756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:02.214349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38590 (local address 127.0.253.254:38127)
0812 16:38:02.214552 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 16:38:02.214562 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:38:02.214599 (+ 37us) server_negotiation.cc:407] Connection header received
0812 16:38:02.215266 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:38:02.215274 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:38:02.215329 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:38:02.215451 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:38:02.216333 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:02.217284 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:02.218371 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:02.218909 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:02.219055 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:38:02.221877 (+ 2822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:38:02.221881 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:38:02.221882 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:38:02.221894 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:38:02.224302 (+ 2408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:38:02.224815 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:38:02.224818 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:38:02.224818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:38:02.224949 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:38:02.225184 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:38:02.225185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:38:02.225185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:38:02.225493 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:38:02.225680 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0812 16:38:02.226172 (+ 492us) server_negotiation.cc:299] Negotiation successful
0812 16:38:02.226328 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":111,"threads_started":1}
I20260812 16:38:02.227375 19774 catalog_manager.cc:1347] Generated new cluster ID: 31cc4a2ded324db29f47f03715799c69
I20260812 16:38:02.227425 19774 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:38:02.240907 19774 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:38:02.241496 19774 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:38:02.248148 19774 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43479def7a5140d9aaaee178f204f7bc: Generated new TSK 0
I20260812 16:38:02.248397 19774 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab.
Entry for principal kudu/127.0.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab.
I20260812 16:38:02.278704 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.193:0
--local_ip_for_outbound_sockets=127.0.253.193
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:38127
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:34419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:38:02.383854 19794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:38:02.384089 19794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:38:02.384117 19794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:38:02.385334 19794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:38:02.385373 19794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:38:02.385402 19794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.193
I20260812 16:38:02.386958 19794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:34419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:38127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.193
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:38:02.387250 19794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:38:02.387502 19794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:38:02.393071 19805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:38:02.393085 19804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:38:02.393090 19810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:38:02.393618 19794 server_base.cc:1034] running on GCE node
I20260812 16:38:02.393841 19794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:38:02.394091 19794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:38:02.395262 19794 hybrid_clock.cc:648] HybridClock initialized: now 1786552682395229 us; error 49 us; skew 500 ppm
Aug 12 16:38:02 dist-test-slave-w4v5 krb5kdc[19621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552682, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:38:02.398136 19794 init.cc:377] Logged in from keytab as kudu/127.0.253.193@KRBTEST.COM (short username kudu)
I20260812 16:38:02.398645 19794 webserver.cc:460] Webserver started at http://127.0.253.193:45959/ using document root <none> and password file <none>
I20260812 16:38:02.398890 19794 fs_manager.cc:359] Metadata directory not provided
I20260812 16:38:02.398960 19794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:38:02.399150 19794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:38:02.400255 19794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/data/instance:
uuid: "d600501eae0144f1b2da7ef6d6e25923"
format_stamp: "Formatted at 2026-08-12 16:38:02 on dist-test-slave-w4v5"
server_key: "4f99530d8e0d1f169d4e8fa48c84b273"
server_key_iv: "39f4938d9121502dc7819aa76dfe05d3"
server_key_version: "encryptionkey@0"
I20260812 16:38:02.400636 19794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance:
uuid: "d600501eae0144f1b2da7ef6d6e25923"
format_stamp: "Formatted at 2026-08-12 16:38:02 on dist-test-slave-w4v5"
server_key: "4f99530d8e0d1f169d4e8fa48c84b273"
server_key_iv: "39f4938d9121502dc7819aa76dfe05d3"
server_key_version: "encryptionkey@0"
I20260812 16:38:02.402019 19794 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:38:02.402844 19829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:38:02.403157 19794 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:38:02.403268 19794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/wal
uuid: "d600501eae0144f1b2da7ef6d6e25923"
format_stamp: "Formatted at 2026-08-12 16:38:02 on dist-test-slave-w4v5"
server_key: "4f99530d8e0d1f169d4e8fa48c84b273"
server_key_iv: "39f4938d9121502dc7819aa76dfe05d3"
server_key_version: "encryptionkey@0"
I20260812 16:38:02.403362 19794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:38:02.436978 19794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:38:02.439045 19794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:38:02.439289 19794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:38:02.439555 19794 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:38:02.439927 19794 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:38:02.439975 19794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:38:02.439997 19794 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:38:02.440011 19794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:38:02.446206 19794 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.193:35961
I20260812 16:38:02.446255 20040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.193:35961 every 8 connection(s)
I20260812 16:38:02.446643 19794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
Aug 12 16:38:02 dist-test-slave-w4v5 krb5kdc[19621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552682, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:38:02.456252 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 19794
I20260812 16:38:02.456369 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance
I20260812 16:38:02.456557 1015 external_mini_cluster.cc:1428] Setting key 65b37927a427353cb764a58ea6ae9859
I20260812 16:38:02.456977 19756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:02.447534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:36003 (local address 127.0.253.254:38127)
0812 16:38:02.447641 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0812 16:38:02.447644 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:38:02.447800 (+ 156us) server_negotiation.cc:407] Connection header received
0812 16:38:02.448326 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:38:02.448327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:38:02.448337 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:38:02.448395 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:38:02.449462 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:02.450032 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:02.451186 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:02.451401 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:02.451461 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:38:02.453783 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:38:02.453785 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:38:02.453785 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:38:02.453793 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:38:02.455361 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:38:02.455766 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:38:02.455767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:38:02.455767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:38:02.455825 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:38:02.456138 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:38:02.456139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:38:02.456140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:38:02.456320 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:38:02.456403 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 16:38:02.456772 (+ 369us) server_negotiation.cc:299] Negotiation successful
0812 16:38:02.456871 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 16:38:02.457207 20044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:02.447471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:38127 (local address 127.0.253.193:36003)
0812 16:38:02.447664 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 16:38:02.447707 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 16:38:02.448242 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:38:02.448453 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:38:02.448457 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:38:02.449136 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:38:02.449341 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:38:02.449346 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:38:02.450147 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:38:02.450148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:38:02.451014 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:38:02.451016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:38:02.451484 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:38:02.451484 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:38:02.451627 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:38:02.452174 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:38:02.452183 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:38:02.453671 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:38:02.455437 (+ 1766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:38:02.455445 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:38:02.455447 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:38:02.455688 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:38:02.455914 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:38:02.455916 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:38:02.455916 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:38:02.456056 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:38:02.456420 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:38:02.456429 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:38:02.456611 (+ 182us) client_negotiation.cc:769] Sending connection context
0812 16:38:02.456740 (+ 129us) client_negotiation.cc:240] Negotiation successful
0812 16:38:02.456954 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":113,"threads_started":1}
I20260812 16:38:02.457741 20041 heartbeater.cc:347] Connected to a master server at 127.0.253.254:38127
I20260812 16:38:02.457850 20041 heartbeater.cc:464] Registering TS with master...
I20260812 16:38:02.458053 20041 heartbeater.cc:511] Master 127.0.253.254:38127 requested a full tablet report, sending...
I20260812 16:38:02.458603 19690 ts_manager.cc:194] Registered new tserver with Master: d600501eae0144f1b2da7ef6d6e25923 (127.0.253.193:35961)
I20260812 16:38:02.459421 19690 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.193@KRBTEST.COM'} at 127.0.253.193:36003
WARNING: no policy specified for kudu/127.0.253.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab.
Entry for principal kudu/127.0.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab.
I20260812 16:38:02.485735 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.194:0
--local_ip_for_outbound_sockets=127.0.253.194
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:38127
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:34419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:38:02.572898 20049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:38:02.573166 20049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:38:02.573207 20049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:38:02.574386 20049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:38:02.574460 20049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:38:02.574505 20049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.194
I20260812 16:38:02.576061 20049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:34419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:38127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.194
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:38:02.576356 20049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:38:02.576623 20049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:38:02.582230 20064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:38:02.582230 20068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:38:02.582258 20062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:38:02.582388 20049 server_base.cc:1034] running on GCE node
I20260812 16:38:02.582700 20049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:38:02.582969 20049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:38:02.584120 20049 hybrid_clock.cc:648] HybridClock initialized: now 1786552682584108 us; error 28 us; skew 500 ppm
Aug 12 16:38:02 dist-test-slave-w4v5 krb5kdc[19621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552682, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:38:02.586926 20049 init.cc:377] Logged in from keytab as kudu/127.0.253.194@KRBTEST.COM (short username kudu)
I20260812 16:38:02.587453 20049 webserver.cc:460] Webserver started at http://127.0.253.194:43103/ using document root <none> and password file <none>
I20260812 16:38:02.587697 20049 fs_manager.cc:359] Metadata directory not provided
I20260812 16:38:02.587767 20049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:38:02.587935 20049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:38:02.589308 20049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/data/instance:
uuid: "4f7688f257a84ceca9c3e52ff44dd273"
format_stamp: "Formatted at 2026-08-12 16:38:02 on dist-test-slave-w4v5"
server_key: "68cb860d3faa88a8bbe84745a085e8e4"
server_key_iv: "c2b96d0e73e983ff8520f181bf67906d"
server_key_version: "encryptionkey@0"
I20260812 16:38:02.589692 20049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance:
uuid: "4f7688f257a84ceca9c3e52ff44dd273"
format_stamp: "Formatted at 2026-08-12 16:38:02 on dist-test-slave-w4v5"
server_key: "68cb860d3faa88a8bbe84745a085e8e4"
server_key_iv: "c2b96d0e73e983ff8520f181bf67906d"
server_key_version: "encryptionkey@0"
I20260812 16:38:02.591203 20049 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:38:02.592069 20082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:38:02.592252 20049 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:38:02.592365 20049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/wal
uuid: "4f7688f257a84ceca9c3e52ff44dd273"
format_stamp: "Formatted at 2026-08-12 16:38:02 on dist-test-slave-w4v5"
server_key: "68cb860d3faa88a8bbe84745a085e8e4"
server_key_iv: "c2b96d0e73e983ff8520f181bf67906d"
server_key_version: "encryptionkey@0"
I20260812 16:38:02.592440 20049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:38:02.620105 20049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:38:02.622350 20049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:38:02.622514 20049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:38:02.622779 20049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:38:02.623214 20049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:38:02.623274 20049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:38:02.623318 20049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:38:02.623347 20049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:38:02.629362 20049 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.194:46057
I20260812 16:38:02.629467 20288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.194:46057 every 8 connection(s)
I20260812 16:38:02.629782 20049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
I20260812 16:38:02.632309 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 20049
I20260812 16:38:02.632426 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance
I20260812 16:38:02.632654 1015 external_mini_cluster.cc:1428] Setting key 42e1ac271580a28291c26d6f8aafc2ce
Aug 12 16:38:02 dist-test-slave-w4v5 krb5kdc[19621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552682, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:38:02.640851 19756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:02.630607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:51769 (local address 127.0.253.254:38127)
0812 16:38:02.630690 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 16:38:02.630692 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:38:02.630903 (+ 211us) server_negotiation.cc:407] Connection header received
0812 16:38:02.631482 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:38:02.631483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:38:02.631494 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:38:02.631553 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:38:02.632569 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:02.633300 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:02.634783 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:02.634977 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:02.635041 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:38:02.637635 (+ 2594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:38:02.637636 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:38:02.637637 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:38:02.637645 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:38:02.639402 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:38:02.639807 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:38:02.639808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:38:02.639809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:38:02.639879 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:38:02.640124 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:38:02.640125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:38:02.640126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:38:02.640291 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:38:02.640377 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 16:38:02.640650 (+ 273us) server_negotiation.cc:299] Negotiation successful
0812 16:38:02.640742 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 16:38:02.640923 20296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:02.630574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:38127 (local address 127.0.253.194:51769)
0812 16:38:02.630772 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 16:38:02.630814 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 16:38:02.631393 (+ 579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:38:02.631609 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:38:02.631612 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:38:02.632205 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:38:02.632402 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:38:02.632433 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:38:02.633399 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:38:02.633400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:38:02.634684 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:38:02.634688 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:38:02.635069 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:38:02.635086 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:38:02.635257 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:38:02.635891 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:38:02.635899 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:38:02.637532 (+ 1633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:38:02.639485 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:38:02.639492 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:38:02.639494 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:38:02.639726 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:38:02.639955 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:38:02.639956 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:38:02.639956 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:38:02.640053 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:38:02.640381 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:38:02.640387 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:38:02.640506 (+ 119us) client_negotiation.cc:769] Sending connection context
0812 16:38:02.640596 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 16:38:02.640733 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":115,"threads_started":1}
I20260812 16:38:02.641441 20290 heartbeater.cc:347] Connected to a master server at 127.0.253.254:38127
I20260812 16:38:02.641574 20290 heartbeater.cc:464] Registering TS with master...
I20260812 16:38:02.641785 20290 heartbeater.cc:511] Master 127.0.253.254:38127 requested a full tablet report, sending...
I20260812 16:38:02.642194 19690 ts_manager.cc:194] Registered new tserver with Master: 4f7688f257a84ceca9c3e52ff44dd273 (127.0.253.194:46057)
I20260812 16:38:02.642580 19690 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.194@KRBTEST.COM'} at 127.0.253.194:51769
WARNING: no policy specified for kudu/127.0.253.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab.
Entry for principal kudu/127.0.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab.
I20260812 16:38:02.663836 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.195:0
--local_ip_for_outbound_sockets=127.0.253.195
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:38127
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:34419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:38:02.750355 20300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:38:02.750586 20300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:38:02.750612 20300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:38:02.751827 20300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:38:02.751870 20300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:38:02.751897 20300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.195
I20260812 16:38:02.753371 20300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:34419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:38127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.195
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:38:02.753618 20300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:38:02.753863 20300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:38:02.759268 20308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:38:02.759413 20307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:38:02.759243 20314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:38:02.759672 20300 server_base.cc:1034] running on GCE node
I20260812 16:38:02.759912 20300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:38:02.760174 20300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:38:02.761341 20300 hybrid_clock.cc:648] HybridClock initialized: now 1786552682761316 us; error 43 us; skew 500 ppm
Aug 12 16:38:02 dist-test-slave-w4v5 krb5kdc[19621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552682, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:38:02.764748 20300 init.cc:377] Logged in from keytab as kudu/127.0.253.195@KRBTEST.COM (short username kudu)
I20260812 16:38:02.765395 20300 webserver.cc:460] Webserver started at http://127.0.253.195:34805/ using document root <none> and password file <none>
I20260812 16:38:02.765746 20300 fs_manager.cc:359] Metadata directory not provided
I20260812 16:38:02.765823 20300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:38:02.765983 20300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:38:02.767390 20300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/data/instance:
uuid: "2742d9eedb6c4c9c8b619aee6e8b94a4"
format_stamp: "Formatted at 2026-08-12 16:38:02 on dist-test-slave-w4v5"
server_key: "78dc445c310526040c6abfe174ecd65e"
server_key_iv: "1b79a123ff850d10de6e861aa5fc4c96"
server_key_version: "encryptionkey@0"
I20260812 16:38:02.767845 20300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance:
uuid: "2742d9eedb6c4c9c8b619aee6e8b94a4"
format_stamp: "Formatted at 2026-08-12 16:38:02 on dist-test-slave-w4v5"
server_key: "78dc445c310526040c6abfe174ecd65e"
server_key_iv: "1b79a123ff850d10de6e861aa5fc4c96"
server_key_version: "encryptionkey@0"
I20260812 16:38:02.769372 20300 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:38:02.770118 20325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:38:02.770385 20300 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:38:02.770515 20300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/wal
uuid: "2742d9eedb6c4c9c8b619aee6e8b94a4"
format_stamp: "Formatted at 2026-08-12 16:38:02 on dist-test-slave-w4v5"
server_key: "78dc445c310526040c6abfe174ecd65e"
server_key_iv: "1b79a123ff850d10de6e861aa5fc4c96"
server_key_version: "encryptionkey@0"
I20260812 16:38:02.770601 20300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:38:02.782002 20300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:38:02.784639 20300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:38:02.784825 20300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:38:02.785104 20300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:38:02.785454 20300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:38:02.785490 20300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:38:02.785545 20300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:38:02.785575 20300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:38:02.792078 20300 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.195:35429
I20260812 16:38:02.792176 20511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.195:35429 every 8 connection(s)
I20260812 16:38:02.792514 20300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
Aug 12 16:38:02 dist-test-slave-w4v5 krb5kdc[19621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552682, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:38:02.800967 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 20300
I20260812 16:38:02.801057 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance
I20260812 16:38:02.801256 1015 external_mini_cluster.cc:1428] Setting key 52f66e761b2f0c2e264095cb5ec6fc74
I20260812 16:38:02.802685 19756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:02.793387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:58203 (local address 127.0.253.254:38127)
0812 16:38:02.793504 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 16:38:02.793507 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:38:02.793679 (+ 172us) server_negotiation.cc:407] Connection header received
0812 16:38:02.794292 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:38:02.794293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:38:02.794307 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:38:02.794361 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:38:02.795583 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:02.796105 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:02.797263 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:02.797474 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:02.797563 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:38:02.799841 (+ 2278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:38:02.799843 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:38:02.799843 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:38:02.799854 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:38:02.801131 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:38:02.801685 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:38:02.801686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:38:02.801686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:38:02.801738 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:38:02.802021 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:38:02.802022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:38:02.802022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:38:02.802205 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:38:02.802300 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 16:38:02.802553 (+ 253us) server_negotiation.cc:299] Negotiation successful
0812 16:38:02.802623 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 16:38:02.802874 20517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:02.793337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:38127 (local address 127.0.253.195:58203)
0812 16:38:02.793540 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 16:38:02.793589 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 16:38:02.794178 (+ 589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:38:02.794450 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:38:02.794453 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:38:02.795183 (+ 730us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:38:02.795488 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:38:02.795494 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:38:02.796209 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:38:02.796210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:38:02.797159 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:38:02.797162 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:38:02.797573 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:38:02.797574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:38:02.797713 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:38:02.798246 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:38:02.798255 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:38:02.799734 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:38:02.801215 (+ 1481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:38:02.801222 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:38:02.801225 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:38:02.801555 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:38:02.801817 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:38:02.801818 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:38:02.801818 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:38:02.801953 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:38:02.802284 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:38:02.802291 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:38:02.802427 (+ 136us) client_negotiation.cc:769] Sending connection context
0812 16:38:02.802518 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 16:38:02.802644 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":87,"threads_started":1}
I20260812 16:38:02.803417 20513 heartbeater.cc:347] Connected to a master server at 127.0.253.254:38127
I20260812 16:38:02.803532 20513 heartbeater.cc:464] Registering TS with master...
I20260812 16:38:02.803731 20513 heartbeater.cc:511] Master 127.0.253.254:38127 requested a full tablet report, sending...
I20260812 16:38:02.804152 19690 ts_manager.cc:194] Registered new tserver with Master: 2742d9eedb6c4c9c8b619aee6e8b94a4 (127.0.253.195:35429)
I20260812 16:38:02.804543 19690 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.195@KRBTEST.COM'} at 127.0.253.195:58203
I20260812 16:38:02.813287 1015 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:38:02.821655 19756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:02.814824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38604 (local address 127.0.253.254:38127)
0812 16:38:02.814910 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0812 16:38:02.814913 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:38:02.815018 (+ 105us) server_negotiation.cc:407] Connection header received
0812 16:38:02.815139 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:38:02.815140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:38:02.815152 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:38:02.815210 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:38:02.815885 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:02.816399 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:02.817158 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:02.817362 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:02.817434 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:38:02.818744 (+ 1310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:38:02.818746 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:38:02.818746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:38:02.818754 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:38:02.820347 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:38:02.820751 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:38:02.820752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:38:02.820753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:38:02.820821 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:38:02.821033 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:38:02.821034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:38:02.821034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:38:02.821215 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:38:02.821331 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 16:38:02.821455 (+ 124us) server_negotiation.cc:299] Negotiation successful
0812 16:38:02.821544 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 16:38:02.823401 19690 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38604:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:38:02.825402 19690 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:38:02.833261 20542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:02.828341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:35961 (local address 127.0.0.1:52652)
0812 16:38:02.828532 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 16:38:02.828544 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:38:02.828601 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:38:02.828983 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:38:02.828984 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:38:02.828990 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:38:02.829074 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:38:02.829076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:38:02.830474 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:38:02.830475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:38:02.831912 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:38:02.831915 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:38:02.832906 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:38:02.832907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:38:02.832990 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:38:02.833003 (+ 13us) client_negotiation.cc:769] Sending connection context
0812 16:38:02.833080 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 16:38:02.833139 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":14,"thread_start_us":80,"threads_started":1}
I20260812 16:38:02.833463 20543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:02.828240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60720 (local address 127.0.253.195:35429)
0812 16:38:02.828806 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0812 16:38:02.828810 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:38:02.828820 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:38:02.828857 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:38:02.828859 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:38:02.828873 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:38:02.828951 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:38:02.829186 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:02.830345 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:02.832043 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:02.832638 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:02.832727 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:38:02.832790 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 16:38:02.833320 (+ 530us) server_negotiation.cc:299] Negotiation successful
0812 16:38:02.833373 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":93,"threads_started":1}
I20260812 16:38:02.833278 20545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:02.828124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.195:35429 (local address 127.0.0.1:60720)
0812 16:38:02.828476 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0812 16:38:02.828487 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:38:02.828531 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:38:02.828980 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:38:02.828983 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:38:02.828986 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:38:02.829099 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:38:02.829100 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:38:02.830473 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:38:02.830474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:38:02.831913 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:38:02.831915 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:38:02.833010 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:38:02.833011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:38:02.833088 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:38:02.833093 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:38:02.833143 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 16:38:02.833191 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":30,"thread_start_us":113,"threads_started":1}
I20260812 16:38:02.833714 20546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:02.828568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52652 (local address 127.0.253.193:35961)
0812 16:38:02.828778 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 16:38:02.828784 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:38:02.828794 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:38:02.828840 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:38:02.828842 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:38:02.828859 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:38:02.829168 (+ 309us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:38:02.829248 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:02.830332 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:02.832010 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:02.832765 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:02.832871 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:38:02.832936 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 16:38:02.833249 (+ 313us) server_negotiation.cc:299] Negotiation successful
0812 16:38:02.833300 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":101,"threads_started":1}
I20260812 16:38:02.833796 20544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:02.828313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45438 (local address 127.0.253.194:46057)
0812 16:38:02.828723 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0812 16:38:02.828728 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:38:02.828739 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:38:02.828783 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:38:02.828785 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:38:02.828802 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:38:02.828884 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:38:02.830475 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:02.831413 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:02.832523 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:02.833271 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:02.833334 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:38:02.833391 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 16:38:02.833673 (+ 282us) server_negotiation.cc:299] Negotiation successful
0812 16:38:02.833717 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":110,"threads_started":1}
I20260812 16:38:02.833815 20540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:02.828113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:46057 (local address 127.0.0.1:45438)
0812 16:38:02.828304 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 16:38:02.828350 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 16:38:02.828419 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:38:02.828980 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:38:02.828983 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:38:02.828988 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:38:02.829098 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:38:02.829100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:38:02.831513 (+ 2413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:38:02.831514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:38:02.832424 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:38:02.832427 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:38:02.833430 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:38:02.833431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:38:02.833498 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:38:02.833502 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:38:02.833547 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 16:38:02.833583 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":6,"thread_start_us":79,"threads_started":1}
I20260812 16:38:02.834474 20398 tablet_service.cc:1467] Processing CreateTablet for tablet c6b465c446954b78882dcdc03acbf13d (DEFAULT_TABLE table=test-table [id=0691512405a3432ea19dff599052e7bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:38:02.834594 19923 tablet_service.cc:1467] Processing CreateTablet for tablet c6b465c446954b78882dcdc03acbf13d (DEFAULT_TABLE table=test-table [id=0691512405a3432ea19dff599052e7bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:38:02.834887 19923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6b465c446954b78882dcdc03acbf13d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:38:02.834887 20398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6b465c446954b78882dcdc03acbf13d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:38:02.835484 20166 tablet_service.cc:1467] Processing CreateTablet for tablet c6b465c446954b78882dcdc03acbf13d (DEFAULT_TABLE table=test-table [id=0691512405a3432ea19dff599052e7bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:38:02.835863 20166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6b465c446954b78882dcdc03acbf13d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:38:02.838549 20547 tablet_bootstrap.cc:492] T c6b465c446954b78882dcdc03acbf13d P 4f7688f257a84ceca9c3e52ff44dd273: Bootstrap starting.
I20260812 16:38:02.838601 20550 tablet_bootstrap.cc:492] T c6b465c446954b78882dcdc03acbf13d P d600501eae0144f1b2da7ef6d6e25923: Bootstrap starting.
I20260812 16:38:02.839259 20549 tablet_bootstrap.cc:492] T c6b465c446954b78882dcdc03acbf13d P 2742d9eedb6c4c9c8b619aee6e8b94a4: Bootstrap starting.
I20260812 16:38:02.839349 20547 tablet_bootstrap.cc:654] T c6b465c446954b78882dcdc03acbf13d P 4f7688f257a84ceca9c3e52ff44dd273: Neither blocks nor log segments found. Creating new log.
I20260812 16:38:02.839350 20550 tablet_bootstrap.cc:654] T c6b465c446954b78882dcdc03acbf13d P d600501eae0144f1b2da7ef6d6e25923: Neither blocks nor log segments found. Creating new log.
I20260812 16:38:02.839847 20549 tablet_bootstrap.cc:654] T c6b465c446954b78882dcdc03acbf13d P 2742d9eedb6c4c9c8b619aee6e8b94a4: Neither blocks nor log segments found. Creating new log.
I20260812 16:38:02.839972 20550 log.cc:824] T c6b465c446954b78882dcdc03acbf13d P d600501eae0144f1b2da7ef6d6e25923: Log is configured to *not* fsync() on all Append() calls
I20260812 16:38:02.839979 20547 log.cc:824] T c6b465c446954b78882dcdc03acbf13d P 4f7688f257a84ceca9c3e52ff44dd273: Log is configured to *not* fsync() on all Append() calls
I20260812 16:38:02.840456 20549 log.cc:824] T c6b465c446954b78882dcdc03acbf13d P 2742d9eedb6c4c9c8b619aee6e8b94a4: Log is configured to *not* fsync() on all Append() calls
I20260812 16:38:02.840811 20547 tablet_bootstrap.cc:492] T c6b465c446954b78882dcdc03acbf13d P 4f7688f257a84ceca9c3e52ff44dd273: No bootstrap required, opened a new log
I20260812 16:38:02.840811 20550 tablet_bootstrap.cc:492] T c6b465c446954b78882dcdc03acbf13d P d600501eae0144f1b2da7ef6d6e25923: No bootstrap required, opened a new log
I20260812 16:38:02.840888 20547 ts_tablet_manager.cc:1397] T c6b465c446954b78882dcdc03acbf13d P 4f7688f257a84ceca9c3e52ff44dd273: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 16:38:02.840888 20550 ts_tablet_manager.cc:1397] T c6b465c446954b78882dcdc03acbf13d P d600501eae0144f1b2da7ef6d6e25923: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:38:02.842901 20550 raft_consensus.cc:348] T c6b465c446954b78882dcdc03acbf13d P d600501eae0144f1b2da7ef6d6e25923 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f7688f257a84ceca9c3e52ff44dd273" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 46057 } } peers { permanent_uuid: "2742d9eedb6c4c9c8b619aee6e8b94a4" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 35429 } } peers { permanent_uuid: "d600501eae0144f1b2da7ef6d6e25923" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 35961 } }
I20260812 16:38:02.842901 20547 raft_consensus.cc:348] T c6b465c446954b78882dcdc03acbf13d P 4f7688f257a84ceca9c3e52ff44dd273 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f7688f257a84ceca9c3e52ff44dd273" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 46057 } } peers { permanent_uuid: "2742d9eedb6c4c9c8b619aee6e8b94a4" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 35429 } } peers { permanent_uuid: "d600501eae0144f1b2da7ef6d6e25923" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 35961 } }
I20260812 16:38:02.843108 20550 raft_consensus.cc:374] T c6b465c446954b78882dcdc03acbf13d P d600501eae0144f1b2da7ef6d6e25923 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:38:02.843108 20547 raft_consensus.cc:374] T c6b465c446954b78882dcdc03acbf13d P 4f7688f257a84ceca9c3e52ff44dd273 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:38:02.843151 20547 raft_consensus.cc:729] T c6b465c446954b78882dcdc03acbf13d P 4f7688f257a84ceca9c3e52ff44dd273 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f7688f257a84ceca9c3e52ff44dd273, State: Initialized, Role: FOLLOWER
I20260812 16:38:02.843151 20550 raft_consensus.cc:729] T c6b465c446954b78882dcdc03acbf13d P d600501eae0144f1b2da7ef6d6e25923 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d600501eae0144f1b2da7ef6d6e25923, State: Initialized, Role: FOLLOWER
I20260812 16:38:02.843299 20547 consensus_queue.cc:260] T c6b465c446954b78882dcdc03acbf13d P 4f7688f257a84ceca9c3e52ff44dd273 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f7688f257a84ceca9c3e52ff44dd273" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 46057 } } peers { permanent_uuid: "2742d9eedb6c4c9c8b619aee6e8b94a4" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 35429 } } peers { permanent_uuid: "d600501eae0144f1b2da7ef6d6e25923" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 35961 } }
I20260812 16:38:02.843299 20550 consensus_queue.cc:260] T c6b465c446954b78882dcdc03acbf13d P d600501eae0144f1b2da7ef6d6e25923 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f7688f257a84ceca9c3e52ff44dd273" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 46057 } } peers { permanent_uuid: "2742d9eedb6c4c9c8b619aee6e8b94a4" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 35429 } } peers { permanent_uuid: "d600501eae0144f1b2da7ef6d6e25923" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 35961 } }
I20260812 16:38:02.843591 20550 ts_tablet_manager.cc:1428] T c6b465c446954b78882dcdc03acbf13d P d600501eae0144f1b2da7ef6d6e25923: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260812 16:38:02.843591 20547 ts_tablet_manager.cc:1428] T c6b465c446954b78882dcdc03acbf13d P 4f7688f257a84ceca9c3e52ff44dd273: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260812 16:38:02.843729 20549 tablet_bootstrap.cc:492] T c6b465c446954b78882dcdc03acbf13d P 2742d9eedb6c4c9c8b619aee6e8b94a4: No bootstrap required, opened a new log
I20260812 16:38:02.843737 20290 heartbeater.cc:503] Master 127.0.253.254:38127 was elected leader, sending a full tablet report...
I20260812 16:38:02.843787 20549 ts_tablet_manager.cc:1397] T c6b465c446954b78882dcdc03acbf13d P 2742d9eedb6c4c9c8b619aee6e8b94a4: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260812 16:38:02.844184 20041 heartbeater.cc:503] Master 127.0.253.254:38127 was elected leader, sending a full tablet report...
I20260812 16:38:02.845561 20549 raft_consensus.cc:348] T c6b465c446954b78882dcdc03acbf13d P 2742d9eedb6c4c9c8b619aee6e8b94a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f7688f257a84ceca9c3e52ff44dd273" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 46057 } } peers { permanent_uuid: "2742d9eedb6c4c9c8b619aee6e8b94a4" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 35429 } } peers { permanent_uuid: "d600501eae0144f1b2da7ef6d6e25923" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 35961 } }
I20260812 16:38:02.845695 20549 raft_consensus.cc:374] T c6b465c446954b78882dcdc03acbf13d P 2742d9eedb6c4c9c8b619aee6e8b94a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:38:02.845721 20549 raft_consensus.cc:729] T c6b465c446954b78882dcdc03acbf13d P 2742d9eedb6c4c9c8b619aee6e8b94a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2742d9eedb6c4c9c8b619aee6e8b94a4, State: Initialized, Role: FOLLOWER
I20260812 16:38:02.845851 20549 consensus_queue.cc:260] T c6b465c446954b78882dcdc03acbf13d P 2742d9eedb6c4c9c8b619aee6e8b94a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f7688f257a84ceca9c3e52ff44dd273" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 46057 } } peers { permanent_uuid: "2742d9eedb6c4c9c8b619aee6e8b94a4" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 35429 } } peers { permanent_uuid: "d600501eae0144f1b2da7ef6d6e25923" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 35961 } }
I20260812 16:38:02.846123 20549 ts_tablet_manager.cc:1428] T c6b465c446954b78882dcdc03acbf13d P 2742d9eedb6c4c9c8b619aee6e8b94a4: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 16:38:02.846184 20513 heartbeater.cc:503] Master 127.0.253.254:38127 was elected leader, sending a full tablet report...
W20260812 16:38:02.880391 20294 tablet.cc:2369] T c6b465c446954b78882dcdc03acbf13d P 4f7688f257a84ceca9c3e52ff44dd273: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 16:38:02.947451 20042 tablet.cc:2369] T c6b465c446954b78882dcdc03acbf13d P d600501eae0144f1b2da7ef6d6e25923: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 16:38:03.043277 20516 tablet.cc:2369] T c6b465c446954b78882dcdc03acbf13d P 2742d9eedb6c4c9c8b619aee6e8b94a4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:38:03.223722 20558 raft_consensus.cc:482] T c6b465c446954b78882dcdc03acbf13d P 2742d9eedb6c4c9c8b619aee6e8b94a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:38:03.223888 20558 raft_consensus.cc:504] T c6b465c446954b78882dcdc03acbf13d P 2742d9eedb6c4c9c8b619aee6e8b94a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f7688f257a84ceca9c3e52ff44dd273" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 46057 } } peers { permanent_uuid: "2742d9eedb6c4c9c8b619aee6e8b94a4" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 35429 } } peers { permanent_uuid: "d600501eae0144f1b2da7ef6d6e25923" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 35961 } }
I20260812 16:38:03.224361 20558 leader_election.cc:288] T c6b465c446954b78882dcdc03acbf13d P 2742d9eedb6c4c9c8b619aee6e8b94a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f7688f257a84ceca9c3e52ff44dd273 (127.0.253.194:46057), d600501eae0144f1b2da7ef6d6e25923 (127.0.253.193:35961)
I20260812 16:38:03.228641 20544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:03.224790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:49857 (local address 127.0.253.194:46057)
0812 16:38:03.224838 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 16:38:03.224840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:38:03.225031 (+ 191us) server_negotiation.cc:407] Connection header received
0812 16:38:03.225061 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:38:03.225064 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:38:03.225078 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:38:03.225126 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:38:03.225302 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:03.226000 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:03.227603 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:03.228180 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:03.228263 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:38:03.228272 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 16:38:03.228526 (+ 254us) server_negotiation.cc:299] Negotiation successful
0812 16:38:03.228572 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 16:38:03.228688 20546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:03.224750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:45845 (local address 127.0.253.193:35961)
0812 16:38:03.224833 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 16:38:03.224835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:38:03.224846 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:38:03.224880 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:38:03.224881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:38:03.224892 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:38:03.224971 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:38:03.225221 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:03.226025 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:03.227604 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:03.228181 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:03.228263 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:38:03.228272 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 16:38:03.228573 (+ 301us) server_negotiation.cc:299] Negotiation successful
0812 16:38:03.228616 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 16:38:03.228785 20563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:03.224642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:46057 (local address 127.0.253.195:49857)
0812 16:38:03.224914 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0812 16:38:03.224922 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 16:38:03.225005 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:38:03.225152 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:38:03.225153 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:38:03.225155 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:38:03.225218 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:38:03.225220 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:38:03.226137 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:38:03.226139 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:38:03.227471 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:38:03.227474 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:38:03.228309 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:38:03.228310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:38:03.228421 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:38:03.228427 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 16:38:03.228504 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 16:38:03.228561 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":65,"thread_start_us":136,"threads_started":1}
I20260812 16:38:03.228986 20517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:03.224642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:35961 (local address 127.0.253.195:45845)
0812 16:38:03.224709 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0812 16:38:03.224723 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 16:38:03.224788 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:38:03.225004 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:38:03.225005 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:38:03.225007 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:38:03.225128 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:38:03.225132 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:38:03.226148 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:38:03.226148 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:38:03.227471 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:38:03.227474 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:38:03.228309 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:38:03.228310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:38:03.228426 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:38:03.228428 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 16:38:03.228504 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 16:38:03.228562 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":101}
I20260812 16:38:03.229202 20204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6b465c446954b78882dcdc03acbf13d" candidate_uuid: "2742d9eedb6c4c9c8b619aee6e8b94a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f7688f257a84ceca9c3e52ff44dd273" is_pre_election: true
I20260812 16:38:03.229388 20204 raft_consensus.cc:2454] T c6b465c446954b78882dcdc03acbf13d P 4f7688f257a84ceca9c3e52ff44dd273 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2742d9eedb6c4c9c8b619aee6e8b94a4 in term 0.
I20260812 16:38:03.229502 19957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6b465c446954b78882dcdc03acbf13d" candidate_uuid: "2742d9eedb6c4c9c8b619aee6e8b94a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d600501eae0144f1b2da7ef6d6e25923" is_pre_election: true
I20260812 16:38:03.229665 20327 leader_election.cc:302] T c6b465c446954b78882dcdc03acbf13d P 2742d9eedb6c4c9c8b619aee6e8b94a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2742d9eedb6c4c9c8b619aee6e8b94a4, 4f7688f257a84ceca9c3e52ff44dd273; no voters:
I20260812 16:38:03.229717 19957 raft_consensus.cc:2454] T c6b465c446954b78882dcdc03acbf13d P d600501eae0144f1b2da7ef6d6e25923 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2742d9eedb6c4c9c8b619aee6e8b94a4 in term 0.
I20260812 16:38:03.229815 20558 raft_consensus.cc:2781] T c6b465c446954b78882dcdc03acbf13d P 2742d9eedb6c4c9c8b619aee6e8b94a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:38:03.229889 20558 raft_consensus.cc:482] T c6b465c446954b78882dcdc03acbf13d P 2742d9eedb6c4c9c8b619aee6e8b94a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:38:03.229919 20558 raft_consensus.cc:3037] T c6b465c446954b78882dcdc03acbf13d P 2742d9eedb6c4c9c8b619aee6e8b94a4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:38:03.230685 20558 raft_consensus.cc:504] T c6b465c446954b78882dcdc03acbf13d P 2742d9eedb6c4c9c8b619aee6e8b94a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f7688f257a84ceca9c3e52ff44dd273" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 46057 } } peers { permanent_uuid: "2742d9eedb6c4c9c8b619aee6e8b94a4" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 35429 } } peers { permanent_uuid: "d600501eae0144f1b2da7ef6d6e25923" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 35961 } }
I20260812 16:38:03.230902 20558 leader_election.cc:288] T c6b465c446954b78882dcdc03acbf13d P 2742d9eedb6c4c9c8b619aee6e8b94a4 [CANDIDATE]: Term 1 election: Requested vote from peers 4f7688f257a84ceca9c3e52ff44dd273 (127.0.253.194:46057), d600501eae0144f1b2da7ef6d6e25923 (127.0.253.193:35961)
I20260812 16:38:03.231045 19957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6b465c446954b78882dcdc03acbf13d" candidate_uuid: "2742d9eedb6c4c9c8b619aee6e8b94a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d600501eae0144f1b2da7ef6d6e25923"
I20260812 16:38:03.231151 20204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6b465c446954b78882dcdc03acbf13d" candidate_uuid: "2742d9eedb6c4c9c8b619aee6e8b94a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f7688f257a84ceca9c3e52ff44dd273"
I20260812 16:38:03.231181 19957 raft_consensus.cc:3037] T c6b465c446954b78882dcdc03acbf13d P d600501eae0144f1b2da7ef6d6e25923 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:38:03.231252 20204 raft_consensus.cc:3037] T c6b465c446954b78882dcdc03acbf13d P 4f7688f257a84ceca9c3e52ff44dd273 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:38:03.231972 20204 raft_consensus.cc:2454] T c6b465c446954b78882dcdc03acbf13d P 4f7688f257a84ceca9c3e52ff44dd273 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2742d9eedb6c4c9c8b619aee6e8b94a4 in term 1.
I20260812 16:38:03.231976 19957 raft_consensus.cc:2454] T c6b465c446954b78882dcdc03acbf13d P d600501eae0144f1b2da7ef6d6e25923 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2742d9eedb6c4c9c8b619aee6e8b94a4 in term 1.
I20260812 16:38:03.232224 20332 leader_election.cc:302] T c6b465c446954b78882dcdc03acbf13d P 2742d9eedb6c4c9c8b619aee6e8b94a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2742d9eedb6c4c9c8b619aee6e8b94a4, d600501eae0144f1b2da7ef6d6e25923; no voters:
I20260812 16:38:03.232335 20558 raft_consensus.cc:2781] T c6b465c446954b78882dcdc03acbf13d P 2742d9eedb6c4c9c8b619aee6e8b94a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:38:03.232491 20558 raft_consensus.cc:686] T c6b465c446954b78882dcdc03acbf13d P 2742d9eedb6c4c9c8b619aee6e8b94a4 [term 1 LEADER]: Becoming Leader. State: Replica: 2742d9eedb6c4c9c8b619aee6e8b94a4, State: Running, Role: LEADER
I20260812 16:38:03.232596 20558 consensus_queue.cc:237] T c6b465c446954b78882dcdc03acbf13d P 2742d9eedb6c4c9c8b619aee6e8b94a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f7688f257a84ceca9c3e52ff44dd273" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 46057 } } peers { permanent_uuid: "2742d9eedb6c4c9c8b619aee6e8b94a4" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 35429 } } peers { permanent_uuid: "d600501eae0144f1b2da7ef6d6e25923" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 35961 } }
I20260812 16:38:03.233511 19690 catalog_manager.cc:5495] T c6b465c446954b78882dcdc03acbf13d P 2742d9eedb6c4c9c8b619aee6e8b94a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2742d9eedb6c4c9c8b619aee6e8b94a4 (127.0.253.195). New cstate: current_term: 1 leader_uuid: "2742d9eedb6c4c9c8b619aee6e8b94a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f7688f257a84ceca9c3e52ff44dd273" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 46057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2742d9eedb6c4c9c8b619aee6e8b94a4" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 35429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d600501eae0144f1b2da7ef6d6e25923" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20260812 16:38:03.276348 20543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:03.273634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60722 (local address 127.0.253.195:35429)
0812 16:38:03.273740 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0812 16:38:03.273742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:38:03.273775 (+ 33us) server_negotiation.cc:407] Connection header received
0812 16:38:03.273843 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:38:03.273843 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:38:03.273864 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:38:03.273947 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:38:03.274146 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:03.274642 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:03.275506 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:03.275707 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:03.275792 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:38:03.275976 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:38:03.276082 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:38:03.276181 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 16:38:03.276211 (+ 30us) server_negotiation.cc:299] Negotiation successful
0812 16:38:03.276233 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 16:38:03.278285 20204 raft_consensus.cc:1261] T c6b465c446954b78882dcdc03acbf13d P 4f7688f257a84ceca9c3e52ff44dd273 [term 1 FOLLOWER]: Refusing update from remote peer 2742d9eedb6c4c9c8b619aee6e8b94a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:38:03.278298 19957 raft_consensus.cc:1261] T c6b465c446954b78882dcdc03acbf13d P d600501eae0144f1b2da7ef6d6e25923 [term 1 FOLLOWER]: Refusing update from remote peer 2742d9eedb6c4c9c8b619aee6e8b94a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:38:03.278692 20564 consensus_queue.cc:1035] T c6b465c446954b78882dcdc03acbf13d P 2742d9eedb6c4c9c8b619aee6e8b94a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d600501eae0144f1b2da7ef6d6e25923" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:38:03.278836 20558 consensus_queue.cc:1035] T c6b465c446954b78882dcdc03acbf13d P 2742d9eedb6c4c9c8b619aee6e8b94a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f7688f257a84ceca9c3e52ff44dd273" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 46057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:38:03.281324 20568 mvcc.cc:205] Tried to move back new op lower bound from 7317719790704771072 to 7317719790521171968. Current Snapshot: MvccSnapshot[applied={T|T < 7317719790704771072}]
I20260812 16:38:03.281744 20571 mvcc.cc:205] Tried to move back new op lower bound from 7317719790704771072 to 7317719790521171968. Current Snapshot: MvccSnapshot[applied={T|T < 7317719790704771072}]
I20260812 16:38:03.281738 20570 mvcc.cc:205] Tried to move back new op lower bound from 7317719790704771072 to 7317719790521171968. Current Snapshot: MvccSnapshot[applied={T|T < 7317719790704771072}]
I20260812 16:38:03.284657 19690 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:38:03.284812 19690 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:38:03.285684 19690 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 43479def7a5140d9aaaee178f204f7bc: Sending DeleteTablet for 3 replicas of tablet c6b465c446954b78882dcdc03acbf13d
I20260812 16:38:03.286046 20166 tablet_service.cc:1514] Processing DeleteTablet for tablet c6b465c446954b78882dcdc03acbf13d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:38:03 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:45438
I20260812 16:38:03.286074 20398 tablet_service.cc:1514] Processing DeleteTablet for tablet c6b465c446954b78882dcdc03acbf13d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:38:03 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:60720
I20260812 16:38:03.286150 19923 tablet_service.cc:1514] Processing DeleteTablet for tablet c6b465c446954b78882dcdc03acbf13d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:38:03 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:52652
I20260812 16:38:03.286415 20585 tablet_replica.cc:330] T c6b465c446954b78882dcdc03acbf13d P 2742d9eedb6c4c9c8b619aee6e8b94a4: stopping tablet replica
I20260812 16:38:03.286540 20585 raft_consensus.cc:2229] T c6b465c446954b78882dcdc03acbf13d P 2742d9eedb6c4c9c8b619aee6e8b94a4 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:38:03.286613 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 19794
I20260812 16:38:03.287521 20585 raft_consensus.cc:2258] T c6b465c446954b78882dcdc03acbf13d P 2742d9eedb6c4c9c8b619aee6e8b94a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:38:03.287763 20582 tablet_replica.cc:330] T c6b465c446954b78882dcdc03acbf13d P 4f7688f257a84ceca9c3e52ff44dd273: stopping tablet replica
I20260812 16:38:03.287915 20582 raft_consensus.cc:2229] T c6b465c446954b78882dcdc03acbf13d P 4f7688f257a84ceca9c3e52ff44dd273 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:38:03.288064 20582 raft_consensus.cc:2258] T c6b465c446954b78882dcdc03acbf13d P 4f7688f257a84ceca9c3e52ff44dd273 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:38:03.292884 20582 ts_tablet_manager.cc:1905] T c6b465c446954b78882dcdc03acbf13d P 4f7688f257a84ceca9c3e52ff44dd273: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:38:03.295212 20585 ts_tablet_manager.cc:1905] T c6b465c446954b78882dcdc03acbf13d P 2742d9eedb6c4c9c8b619aee6e8b94a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:38:03.296731 20585 ts_tablet_manager.cc:1918] T c6b465c446954b78882dcdc03acbf13d P 2742d9eedb6c4c9c8b619aee6e8b94a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:38:03.296892 20585 log.cc:1196] T c6b465c446954b78882dcdc03acbf13d P 2742d9eedb6c4c9c8b619aee6e8b94a4: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-2/wal/wals/c6b465c446954b78882dcdc03acbf13d
I20260812 16:38:03.297188 20585 ts_tablet_manager.cc:1939] T c6b465c446954b78882dcdc03acbf13d P 2742d9eedb6c4c9c8b619aee6e8b94a4: Deleting consensus metadata
I20260812 16:38:03.298578 20582 ts_tablet_manager.cc:1918] T c6b465c446954b78882dcdc03acbf13d P 4f7688f257a84ceca9c3e52ff44dd273: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:38:03.298691 20582 log.cc:1196] T c6b465c446954b78882dcdc03acbf13d P 4f7688f257a84ceca9c3e52ff44dd273: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552619305671-1015-0/minicluster-data/ts-1/wal/wals/c6b465c446954b78882dcdc03acbf13d
I20260812 16:38:03.299094 20582 ts_tablet_manager.cc:1939] T c6b465c446954b78882dcdc03acbf13d P 4f7688f257a84ceca9c3e52ff44dd273: Deleting consensus metadata
I20260812 16:38:03.299691 19670 catalog_manager.cc:4841] TS 4f7688f257a84ceca9c3e52ff44dd273 (127.0.253.194:46057): tablet c6b465c446954b78882dcdc03acbf13d (table test-table [id=0691512405a3432ea19dff599052e7bf]) successfully deleted
I20260812 16:38:03.303308 19672 catalog_manager.cc:4841] TS 2742d9eedb6c4c9c8b619aee6e8b94a4 (127.0.253.195:35429): tablet c6b465c446954b78882dcdc03acbf13d (table test-table [id=0691512405a3432ea19dff599052e7bf]) successfully deleted
W20260812 16:38:03.310237 19672 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.253.193:35961 (error 108)
I20260812 16:38:03.310568 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 20049
I20260812 16:38:03.316608 20545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:03.310759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:35961 (local address 127.0.0.1:52668)
0812 16:38:03.316478 (+ 5719us) negotiation.cc:107] Waiting for socket to connect
0812 16:38:03.316554 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.193:35961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5681}
W20260812 16:38:03.320497 19672 catalog_manager.cc:4584] TS d600501eae0144f1b2da7ef6d6e25923 (127.0.253.193:35961): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c6b465c446954b78882dcdc03acbf13d: Network error: Client connection negotiation failed: client connection to 127.0.253.193:35961: connect: Connection refused (error 111)
I20260812 16:38:03.324947 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 20300
I20260812 16:38:03.343004 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 19641
2026-08-12T16:38:03Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2714 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:38:03 dist-test-slave-w4v5 krb5kdc[20592](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:38:03 dist-test-slave-w4v5 krb5kdc[20592](info): set up 2 sockets
Aug 12 16:38:03 dist-test-slave-w4v5 krb5kdc[20592](info): commencing operation
krb5kdc: starting...
W20260812 16:38:04.669322 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.293s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:38:04 dist-test-slave-w4v5 krb5kdc[20592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:38:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:38:04Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.0.253.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.0.253.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.0.253.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/krb5kdc/rangeradmin_127.0.253.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.0.253.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/krb5kdc/rangeradmin_127.0.253.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.0.253.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.0.253.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.0.253.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/krb5kdc/rangerlookup_127.0.253.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.0.253.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/krb5kdc/rangerlookup_127.0.253.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.253.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.0.253.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.253.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/krb5kdc/HTTP_127.0.253.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.253.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/krb5kdc/HTTP_127.0.253.212@KRBTEST.COM.keytab.
I20260812 16:38:04.854152 1015 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskJXkJy5/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/postgres -l logfile start
2026-08-12 16:38:07.540 UTC [20643] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 16:38:07.540 UTC [20643] LOG: listening on IPv4 address "127.0.253.212", port 52289
2026-08-12 16:38:07.544 UTC [20643] LOG: listening on Unix socket "/tmp/.s.PGSQL.52289"
2026-08-12 16:38:07.581 UTC [20648] LOG: database system was shut down at 2026-08-12 16:38:06 UTC
2026-08-12 16:38:07.585 UTC [20643] LOG: database system is ready to accept connections
I20260812 16:38:08.860646 1015 mini_postgres.cc:96] Postgres bound to 52289
2026-08-12 16:38:08.869 UTC [20663] FATAL: database "slave" does not exist
127.0.253.212:52289 - accepting connections
I20260812 16:38:08.870503 1015 mini_ranger.cc:161] Starting Ranger...
I20260812 16:38:08.887692 1015 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 16:38:09.403216 1015 mini_ranger.cc:87] Created ranger Postgres database
I20260812 16:38:09.403308 1015 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ranger-admin
2026-08-12 16:38:09,739 [I] DB FLAVOR :POSTGRES
2026-08-12 16:38:09,739 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 16:38:09,739 [I] Checking connection..
2026-08-12 16:38:09,739 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:52289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 16:38:10,021 [I] Checking connection passed.
2026-08-12 16:38:10,022 [I] --------- Verifying version history table ---------
2026-08-12 16:38:10,022 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:52289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 16:38:10,287 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 16:38:10,288 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:52289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 16:38:10,549 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 16:38:10,549 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 16:38:10,549 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:52289/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 16:38:10.824 UTC [20772] WARNING: there is no transaction in progress
2026-08-12 16:38:10,835 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 16:38:10,835 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:52289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 16:38:11,094 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 16:38:11,094 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 16:38:11,095 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:52289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 16:38:11,363 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:52289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 16:38:11,637 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:52289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-w4v5.c.gcp-upstream.internal','N') ;"
2026-08-12 16:38:11,880 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:52289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 16:38:12,157 [I] Table x_portal_user does not exist in database ranger
2026-08-12 16:38:12,158 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:52289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 16:38:12,423 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 16:38:12,423 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:52289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 16:38:12,662 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:52289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 16:38:12,914 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 16:38:12,914 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:52289/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 16:38:13.917 UTC [20974] WARNING: there is no transaction in progress
2026-08-12 16:38:13.934 UTC [20974] WARNING: there is no transaction in progress
2026-08-12 16:38:13.951 UTC [20974] WARNING: there is no transaction in progress
2026-08-12 16:38:13.967 UTC [20974] WARNING: there is no transaction in progress
2026-08-12 16:38:13.986 UTC [20974] WARNING: there is no transaction in progress
2026-08-12 16:38:14.004 UTC [20974] WARNING: there is no transaction in progress
2026-08-12 16:38:14.068 UTC [20974] WARNING: there is no transaction in progress
2026-08-12 16:38:14.075 UTC [20974] WARNING: there is no transaction in progress
2026-08-12 16:38:14.091 UTC [20974] WARNING: there is no transaction in progress
2026-08-12 16:38:14.102 UTC [20974] WARNING: there is no transaction in progress
2026-08-12 16:38:14.113 UTC [20974] WARNING: there is no transaction in progress
2026-08-12 16:38:14.128 UTC [20974] WARNING: there is no transaction in progress
2026-08-12 16:38:14.139 UTC [20974] WARNING: there is no transaction in progress
2026-08-12 16:38:14.146 UTC [20974] WARNING: there is no transaction in progress
2026-08-12 16:38:14.154 UTC [20974] WARNING: there is no transaction in progress
2026-08-12 16:38:14.162 UTC [20974] WARNING: there is no transaction in progress
2026-08-12 16:38:14,851 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 16:38:14,852 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:52289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 16:38:15,124 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 16:38:15,125 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:52289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 16:38:15,368 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 16:38:15,368 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:52289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 16:38:15,629 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 16:38:15,629 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:52289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 16:38:15,878 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:52289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 16:38:16,134 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:52289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-w4v5.c.gcp-upstream.internal';"
2026-08-12 16:38:16,398 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 16:38:16,398 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:52289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 16:38:16,637 [I] DB_PATCHES have already been applied
I20260812 16:38:16.643877 1015 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ranger-admin:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskJXkJy5/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 16:38:16.643944 1015 mini_ranger.cc:193] Using host: 127.0.253.212
I20260812 16:38:16.648017 1015 mini_ranger.cc:227] Ranger admin URL: http://127.0.253.212:36329
Aug 12, 2026 4:38:16 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 4:38:16 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 4:38:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 4:38:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 4:38:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 4:38:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 4:38:17 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 16:38:18.099627 1015 mini_ranger.cc:160] Time spent starting Ranger: real 9.229s user 0.000s sys 0.012s
Aug 12, 2026 4:38:39 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 4:38:39 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Aug 12, 2026 4:38:39 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 4:38:39 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 4:38:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 4:38:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 4:38:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 4:38:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 4:38:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 4:38:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 4:38:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 4:38:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 4:38:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 4:38:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 4:38:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 4:38:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 4:38:39 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 4:38:40 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 16:38:41.009250 1015 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.0.253.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.0.253.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.0.253.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/krb5kdc/rangerkms_127.0.253.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.0.253.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/krb5kdc/rangerkms_127.0.253.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.253.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.0.253.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.253.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/krb5kdc/HTTP_127.0.253.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.253.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/krb5kdc/HTTP_127.0.253.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 16:38:41 dist-test-slave-w4v5 krb5kdc[20592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552721, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 16:38:41.104880 1015 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 16:38:41.115789 1015 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 16:38:42.026947 1015 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 16:38:42.027040 1015 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ranger-kms
I20260812 16:38:42.027063 1015 mini_ranger_kms.cc:227] Using postgres at 127.0.253.212:52289
2026-08-12 16:38:42,260 [I] DB FLAVOR :POSTGRES
2026-08-12 16:38:42,261 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 16:38:42,261 [I] Checking connection
2026-08-12 16:38:42,261 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:52289/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 16:38:42,503 [I] connection success
2026-08-12 16:38:42,503 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 16:38:42,503 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 16:38:42,503 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:52289/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 16:38:42,773 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 16:38:42,773 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 16:38:42,773 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 16:38:42,773 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:52289/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 16:38:43,055 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 16:38:43.561465 1015 mini_ranger_kms.cc:310] Created kms service
I20260812 16:38:43.775184 1015 mini_ranger_kms.cc:326] Created kudu user
I20260812 16:38:43.826349 1015 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 16:38:43 dist-test-slave-w4v5 krb5kdc[20592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552721, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.0.253.212@KRBTEST.COM
I20260812 16:38:44.283290 1015 mini_ranger_kms.cc:384] Added ranger policy
I20260812 16:38:44.283437 1015 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ranger-kms:/tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 16:38:44.283465 1015 mini_ranger_kms.cc:240] Using host: 127.0.253.212
I20260812 16:38:44.287899 1015 mini_ranger_kms.cc:276] Ranger KMS PID: 21340
I20260812 16:38:44.288033 1015 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.0.253.212:39765
Aug 12, 2026 4:38:45 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 4:38:45 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 4:38:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 4:38:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 4:38:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 4:38:45 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 16:38:46 dist-test-slave-w4v5 krb5kdc[20592](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786552726, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.253.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 12 16:38:47 dist-test-slave-w4v5 krb5kdc[20592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552726, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.253.212@KRBTEST.COM for HTTP/127.0.253.212@KRBTEST.COM
I20260812 16:38:47.181473 1015 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.077s user 0.010s sys 0.205s
I20260812 16:38:47.181587 1015 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 16:38:47.181663 1015 mini_ranger_kms.cc:401] 127.0.253.212:39765/kms/v1/keys
Aug 12 16:38:47 dist-test-slave-w4v5 krb5kdc[20592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552726, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.253.212@KRBTEST.COM for HTTP/127.0.253.212@KRBTEST.COM
Aug 12 16:38:49 dist-test-slave-w4v5 krb5kdc[20592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552729, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.0.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
I20260812 16:38:49.897883 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:42711
--webserver_interface=127.0.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:34767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:42711
--ranger_config_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.253.212:39765/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:38:49.986071 21427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:38:49.986331 21427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:38:49.986371 21427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:38:49.987646 21427 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:38:49.987713 21427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:38:49.987738 21427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:38:49.987766 21427 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:38:49.987799 21427 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:38:49.989555 21427 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:34767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/master-0/wal
--ranger_kms_url=127.0.253.212:39765/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:42711
--ranger_config_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:42711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:38:49.989868 21427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:38:49.990121 21427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:38:49.995853 21442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:38:49.995821 21439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:38:49.996019 21427 server_base.cc:1034] running on GCE node
W20260812 16:38:49.995827 21445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:38:49.996387 21427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:38:49.996745 21427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:38:49.997951 21427 hybrid_clock.cc:648] HybridClock initialized: now 1786552729997922 us; error 47 us; skew 500 ppm
Aug 12 16:38:49 dist-test-slave-w4v5 krb5kdc[20592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552729, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:38:50.000815 21427 init.cc:377] Logged in from keytab as kudu/127.0.253.254@KRBTEST.COM (short username kudu)
I20260812 16:38:50.001386 21427 webserver.cc:460] Webserver started at http://127.0.253.254:36881/ using document root <none> and password file <none>
I20260812 16:38:50.001644 21427 fs_manager.cc:359] Metadata directory not provided
I20260812 16:38:50.001686 21427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:38:50.001818 21427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 16:38:50 dist-test-slave-w4v5 krb5kdc[20592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552729, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for HTTP/127.0.253.212@KRBTEST.COM
I20260812 16:38:50.058630 21427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "0a0d9e0a42074ad5a6f0564378f355a9"
format_stamp: "Formatted at 2026-08-12 16:38:50 on dist-test-slave-w4v5"
server_key: "7715dfd03e2abc68c23b3f770541158f"
server_key_iv: "7069551cde6a00b51121c766e9cf24a8"
server_key_version: "kuduclusterkey@0"
I20260812 16:38:50.059355 21427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "0a0d9e0a42074ad5a6f0564378f355a9"
format_stamp: "Formatted at 2026-08-12 16:38:50 on dist-test-slave-w4v5"
server_key: "7715dfd03e2abc68c23b3f770541158f"
server_key_iv: "7069551cde6a00b51121c766e9cf24a8"
server_key_version: "kuduclusterkey@0"
I20260812 16:38:50.061111 21427 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 16:38:50.073945 21459 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:38:50.074239 21427 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:38:50.074364 21427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "0a0d9e0a42074ad5a6f0564378f355a9"
format_stamp: "Formatted at 2026-08-12 16:38:50 on dist-test-slave-w4v5"
server_key: "7715dfd03e2abc68c23b3f770541158f"
server_key_iv: "7069551cde6a00b51121c766e9cf24a8"
server_key_version: "kuduclusterkey@0"
I20260812 16:38:50.074470 21427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:38:50.120398 21427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:38:50.122661 21427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:38:50.122829 21427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:38:50.128113 21427 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:42711
I20260812 16:38:50.128165 21567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:42711 every 8 connection(s)
I20260812 16:38:50.128628 21427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:38:50.129434 21571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:38:50.131917 21571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a0d9e0a42074ad5a6f0564378f355a9: Bootstrap starting.
I20260812 16:38:50.132557 21571 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a0d9e0a42074ad5a6f0564378f355a9: Neither blocks nor log segments found. Creating new log.
I20260812 16:38:50.132870 21571 log.cc:824] T 00000000000000000000000000000000 P 0a0d9e0a42074ad5a6f0564378f355a9: Log is configured to *not* fsync() on all Append() calls
I20260812 16:38:50.133721 21571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a0d9e0a42074ad5a6f0564378f355a9: No bootstrap required, opened a new log
I20260812 16:38:50.135928 21571 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a0d9e0a42074ad5a6f0564378f355a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a0d9e0a42074ad5a6f0564378f355a9" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 42711 } }
I20260812 16:38:50.136106 21571 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a0d9e0a42074ad5a6f0564378f355a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:38:50.136184 21571 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a0d9e0a42074ad5a6f0564378f355a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0d9e0a42074ad5a6f0564378f355a9, State: Initialized, Role: FOLLOWER
I20260812 16:38:50.136380 21571 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a0d9e0a42074ad5a6f0564378f355a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a0d9e0a42074ad5a6f0564378f355a9" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 42711 } }
I20260812 16:38:50.136485 21571 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a0d9e0a42074ad5a6f0564378f355a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:38:50.136530 21571 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a0d9e0a42074ad5a6f0564378f355a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:38:50.136582 21571 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a0d9e0a42074ad5a6f0564378f355a9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:38:50.136849 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 21427
I20260812 16:38:50.136965 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:38:50.137162 1015 external_mini_cluster.cc:1428] Setting key 5d3ff5fa14009642e811155d2f6b3fa5
I20260812 16:38:50.137378 21571 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a0d9e0a42074ad5a6f0564378f355a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a0d9e0a42074ad5a6f0564378f355a9" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 42711 } }
I20260812 16:38:50.137513 21571 leader_election.cc:302] T 00000000000000000000000000000000 P 0a0d9e0a42074ad5a6f0564378f355a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a0d9e0a42074ad5a6f0564378f355a9; no voters:
I20260812 16:38:50.137688 21571 leader_election.cc:288] T 00000000000000000000000000000000 P 0a0d9e0a42074ad5a6f0564378f355a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:38:50.137717 21574 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a0d9e0a42074ad5a6f0564378f355a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:38:50.137956 21574 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a0d9e0a42074ad5a6f0564378f355a9 [term 1 LEADER]: Becoming Leader. State: Replica: 0a0d9e0a42074ad5a6f0564378f355a9, State: Running, Role: LEADER
I20260812 16:38:50.138113 21574 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a0d9e0a42074ad5a6f0564378f355a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a0d9e0a42074ad5a6f0564378f355a9" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 42711 } }
I20260812 16:38:50.138470 21574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a0d9e0a42074ad5a6f0564378f355a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a0d9e0a42074ad5a6f0564378f355a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a0d9e0a42074ad5a6f0564378f355a9" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 42711 } } }
I20260812 16:38:50.138615 21574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a0d9e0a42074ad5a6f0564378f355a9 [sys.catalog]: This master's current role is: LEADER
I20260812 16:38:50.138620 21571 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a0d9e0a42074ad5a6f0564378f355a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:38:50.138455 21582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a0d9e0a42074ad5a6f0564378f355a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a0d9e0a42074ad5a6f0564378f355a9. Latest consensus state: current_term: 1 leader_uuid: "0a0d9e0a42074ad5a6f0564378f355a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a0d9e0a42074ad5a6f0564378f355a9" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 42711 } } }
I20260812 16:38:50.138939 21582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a0d9e0a42074ad5a6f0564378f355a9 [sys.catalog]: This master's current role is: LEADER
I20260812 16:38:50.140406 21571 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 16:38:50.142999 21592 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:38:50.144254 21592 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 16:38:50 dist-test-slave-w4v5 krb5kdc[20592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552729, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:38:50.151873 21589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:50.137920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45284 (local address 127.0.253.254:42711)
0812 16:38:50.138126 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 16:38:50.138136 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:38:50.138168 (+ 32us) server_negotiation.cc:407] Connection header received
0812 16:38:50.138294 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:38:50.138301 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:38:50.138359 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:38:50.139303 (+ 944us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:38:50.139403 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:50.140816 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:50.142413 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:50.143881 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:50.144010 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:38:50.147220 (+ 3210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:38:50.147224 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:38:50.147225 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:38:50.147237 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:38:50.149315 (+ 2078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:38:50.149762 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:38:50.149766 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:38:50.149766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:38:50.149879 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:38:50.150093 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:38:50.150094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:38:50.150094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:38:50.150438 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:38:50.150616 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0812 16:38:50.151197 (+ 581us) server_negotiation.cc:299] Negotiation successful
0812 16:38:50.151336 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":97,"threads_started":1}
I20260812 16:38:50.154049 21592 catalog_manager.cc:1347] Generated new cluster ID: 066bff22393e402ea70ce456942204c3
I20260812 16:38:50.154119 21592 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:38:50.170959 21592 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:38:50.171788 21592 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:38:50.186458 21592 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a0d9e0a42074ad5a6f0564378f355a9: Generated new TSK 0
I20260812 16:38:50.186818 21592 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab.
Entry for principal kudu/127.0.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab.
I20260812 16:38:50.956252 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.253.212:39765/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.193:0
--local_ip_for_outbound_sockets=127.0.253.193
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:42711
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:34767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:38:51.050360 21654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:38:51.050678 21654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:38:51.050719 21654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:38:51.052021 21654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:38:51.052095 21654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:38:51.052143 21654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.193
I20260812 16:38:51.053735 21654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:34767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.0.253.212:39765/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:42711
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.193
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:38:51.054000 21654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:38:51.054248 21654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:38:51.059710 21665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:38:51.059856 21669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:38:51.059742 21664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:38:51.061112 21654 server_base.cc:1034] running on GCE node
I20260812 16:38:51.061353 21654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:38:51.061666 21654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:38:51.062860 21654 hybrid_clock.cc:648] HybridClock initialized: now 1786552731062833 us; error 45 us; skew 500 ppm
Aug 12 16:38:51 dist-test-slave-w4v5 krb5kdc[20592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552731, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:38:51.065737 21654 init.cc:377] Logged in from keytab as kudu/127.0.253.193@KRBTEST.COM (short username kudu)
I20260812 16:38:51.066279 21654 webserver.cc:460] Webserver started at http://127.0.253.193:43751/ using document root <none> and password file <none>
I20260812 16:38:51.066533 21654 fs_manager.cc:359] Metadata directory not provided
I20260812 16:38:51.066581 21654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:38:51.066715 21654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 16:38:51 dist-test-slave-w4v5 krb5kdc[20592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552731, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for HTTP/127.0.253.212@KRBTEST.COM
I20260812 16:38:51.080701 21654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-0/data/instance:
uuid: "89de05aa958645f4b058bb40b1bc794d"
format_stamp: "Formatted at 2026-08-12 16:38:51 on dist-test-slave-w4v5"
server_key: "1ed80667097845f0fd542d1c16d889c4"
server_key_iv: "363f2392df0d2ebb8d0b4333f784ccd8"
server_key_version: "kuduclusterkey@0"
I20260812 16:38:51.081154 21654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance:
uuid: "89de05aa958645f4b058bb40b1bc794d"
format_stamp: "Formatted at 2026-08-12 16:38:51 on dist-test-slave-w4v5"
server_key: "1ed80667097845f0fd542d1c16d889c4"
server_key_iv: "363f2392df0d2ebb8d0b4333f784ccd8"
server_key_version: "kuduclusterkey@0"
I20260812 16:38:51.083106 21654 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 16:38:51.096860 21689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:38:51.097092 21654 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:38:51.097203 21654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-0/wal
uuid: "89de05aa958645f4b058bb40b1bc794d"
format_stamp: "Formatted at 2026-08-12 16:38:51 on dist-test-slave-w4v5"
server_key: "1ed80667097845f0fd542d1c16d889c4"
server_key_iv: "363f2392df0d2ebb8d0b4333f784ccd8"
server_key_version: "kuduclusterkey@0"
I20260812 16:38:51.097329 21654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:38:51.110097 21654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:38:51.113277 21654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:38:51.113550 21654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:38:51.113904 21654 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:38:51.114542 21654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:38:51.114660 21654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:38:51.114768 21654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:38:51.114848 21654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:38:51.122661 21654 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.193:42673
I20260812 16:38:51.123281 21654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
I20260812 16:38:51.127267 21906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.193:42673 every 8 connection(s)
I20260812 16:38:51.129113 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 21654
I20260812 16:38:51.129215 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance
I20260812 16:38:51.129403 1015 external_mini_cluster.cc:1428] Setting key 34f22c4d23526fdad77e07363cf2a3ee
Aug 12 16:38:51 dist-test-slave-w4v5 krb5kdc[20592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552731, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:38:51.142516 21910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:51.124185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:42711 (local address 127.0.253.193:55149)
0812 16:38:51.124469 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0812 16:38:51.124520 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 16:38:51.125213 (+ 693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:38:51.125677 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:38:51.125687 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:38:51.125970 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:38:51.126266 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:38:51.126273 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:38:51.127338 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:38:51.127339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:38:51.128738 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:38:51.128741 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:38:51.129268 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:38:51.129269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:38:51.129416 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:38:51.130069 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:38:51.130080 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:38:51.136776 (+ 6696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:38:51.140631 (+ 3855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:38:51.140642 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:38:51.140645 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:38:51.140997 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:38:51.141268 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:38:51.141269 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:38:51.141269 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:38:51.141429 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:38:51.141759 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:38:51.141766 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:38:51.141949 (+ 183us) client_negotiation.cc:769] Sending connection context
0812 16:38:51.142068 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 16:38:51.142252 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":88,"threads_started":1}
I20260812 16:38:51.143370 21913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:51.124215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:55149 (local address 127.0.253.254:42711)
0812 16:38:51.125424 (+ 1209us) server_negotiation.cc:206] Beginning negotiation
0812 16:38:51.125426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:38:51.125436 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:38:51.125463 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:38:51.125464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:38:51.125478 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:38:51.125553 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:38:51.126408 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:51.127217 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:51.128836 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:51.129135 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:51.129198 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:38:51.138430 (+ 9232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:38:51.138432 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:38:51.138432 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:38:51.138443 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:38:51.140506 (+ 2063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:38:51.141115 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:38:51.141116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:38:51.141117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:38:51.141187 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:38:51.141514 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:38:51.141514 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:38:51.141515 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:38:51.141670 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:38:51.143129 (+ 1459us) server_negotiation.cc:1035] Waiting for connection context
0812 16:38:51.143178 (+ 49us) server_negotiation.cc:299] Negotiation successful
0812 16:38:51.143273 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1175,"thread_start_us":80,"threads_started":1}
I20260812 16:38:51.144047 21907 heartbeater.cc:347] Connected to a master server at 127.0.253.254:42711
I20260812 16:38:51.144181 21907 heartbeater.cc:464] Registering TS with master...
I20260812 16:38:51.144409 21907 heartbeater.cc:511] Master 127.0.253.254:42711 requested a full tablet report, sending...
I20260812 16:38:51.145041 21496 ts_manager.cc:194] Registered new tserver with Master: 89de05aa958645f4b058bb40b1bc794d (127.0.253.193:42673)
I20260812 16:38:51.145944 21496 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.193@KRBTEST.COM'} at 127.0.253.193:55149
WARNING: no policy specified for kudu/127.0.253.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab.
Entry for principal kudu/127.0.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab.
I20260812 16:38:51.169755 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.253.212:39765/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.194:0
--local_ip_for_outbound_sockets=127.0.253.194
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:42711
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:34767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:38:51.283422 21927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:38:51.283834 21927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:38:51.283937 21927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:38:51.285852 21927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:38:51.285981 21927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:38:51.286075 21927 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.194
I20260812 16:38:51.287848 21927 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:34767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.0.253.212:39765/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:42711
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21927
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.194
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:38:51.288132 21927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:38:51.288408 21927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:38:51.293823 21946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:38:51.294183 21940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:38:51.293825 21939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:38:51.294824 21927 server_base.cc:1034] running on GCE node
I20260812 16:38:51.295051 21927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:38:51.295379 21927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:38:51.296569 21927 hybrid_clock.cc:648] HybridClock initialized: now 1786552731296550 us; error 37 us; skew 500 ppm
Aug 12 16:38:51 dist-test-slave-w4v5 krb5kdc[20592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552731, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:38:51.299409 21927 init.cc:377] Logged in from keytab as kudu/127.0.253.194@KRBTEST.COM (short username kudu)
I20260812 16:38:51.300005 21927 webserver.cc:460] Webserver started at http://127.0.253.194:44559/ using document root <none> and password file <none>
I20260812 16:38:51.300249 21927 fs_manager.cc:359] Metadata directory not provided
I20260812 16:38:51.300320 21927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:38:51.300477 21927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 16:38:51 dist-test-slave-w4v5 krb5kdc[20592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552731, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for HTTP/127.0.253.212@KRBTEST.COM
I20260812 16:38:51.320153 21927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-1/data/instance:
uuid: "7aed80c2ca324bf09fae723cad2eb10b"
format_stamp: "Formatted at 2026-08-12 16:38:51 on dist-test-slave-w4v5"
server_key: "a55c7f851a907596273796bdc1d4d7d5"
server_key_iv: "43cf1cb0b7fa89205f0c21cd70be09e0"
server_key_version: "kuduclusterkey@0"
I20260812 16:38:51.320734 21927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance:
uuid: "7aed80c2ca324bf09fae723cad2eb10b"
format_stamp: "Formatted at 2026-08-12 16:38:51 on dist-test-slave-w4v5"
server_key: "a55c7f851a907596273796bdc1d4d7d5"
server_key_iv: "43cf1cb0b7fa89205f0c21cd70be09e0"
server_key_version: "kuduclusterkey@0"
I20260812 16:38:51.327565 21927 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.000s
I20260812 16:38:51.347271 21958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:38:51.347575 21927 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260812 16:38:51.347687 21927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-1/wal
uuid: "7aed80c2ca324bf09fae723cad2eb10b"
format_stamp: "Formatted at 2026-08-12 16:38:51 on dist-test-slave-w4v5"
server_key: "a55c7f851a907596273796bdc1d4d7d5"
server_key_iv: "43cf1cb0b7fa89205f0c21cd70be09e0"
server_key_version: "kuduclusterkey@0"
I20260812 16:38:51.347798 21927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 12 16:38:51 dist-test-slave-w4v5 krb5kdc[20592](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786552731, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:38:51.368311 21927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:38:51.371263 21927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:38:51.371505 21927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:38:51.371860 21927 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:38:51.372313 21927 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:38:51.372409 21927 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:38:51.372490 21927 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:38:51.372572 21927 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:38:51.379851 21927 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.194:43395
I20260812 16:38:51.380380 21927 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
I20260812 16:38:51.381059 22190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.194:43395 every 8 connection(s)
I20260812 16:38:51.390168 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 21927
I20260812 16:38:51.390300 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance
I20260812 16:38:51.390538 1015 external_mini_cluster.cc:1428] Setting key 8f7655af30ba5fbc0d1dbc97ebfefdff
Aug 12 16:38:51 dist-test-slave-w4v5 krb5kdc[20592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552731, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:38:51.401854 22195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:51.381872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:42711 (local address 127.0.253.194:49541)
0812 16:38:51.386181 (+ 4309us) negotiation.cc:107] Waiting for socket to connect
0812 16:38:51.386237 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 16:38:51.386962 (+ 725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:38:51.387419 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:38:51.387424 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:38:51.387722 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:38:51.388039 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:38:51.388046 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:38:51.389141 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:38:51.389142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:38:51.390472 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:38:51.390476 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:38:51.395300 (+ 4824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:38:51.395302 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:38:51.395490 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:38:51.396160 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:38:51.396172 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:38:51.397893 (+ 1721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:38:51.399870 (+ 1977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:38:51.399878 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:38:51.399881 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:38:51.400289 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:38:51.400520 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:38:51.400521 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:38:51.400521 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:38:51.400682 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:38:51.401063 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:38:51.401070 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:38:51.401246 (+ 176us) client_negotiation.cc:769] Sending connection context
0812 16:38:51.401409 (+ 163us) client_negotiation.cc:240] Negotiation successful
0812 16:38:51.401599 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4248,"thread_start_us":95,"threads_started":1}
I20260812 16:38:51.402405 21913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:51.382274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:49541 (local address 127.0.253.254:42711)
0812 16:38:51.382321 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 16:38:51.382324 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:38:51.386352 (+ 4028us) server_negotiation.cc:407] Connection header received
0812 16:38:51.387139 (+ 787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:38:51.387141 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:38:51.387157 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:38:51.387197 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:38:51.388170 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:51.388968 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:51.392652 (+ 3684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:51.392983 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:51.393061 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:38:51.398013 (+ 4952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:38:51.398016 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:38:51.398016 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:38:51.398025 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:38:51.399752 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:38:51.400380 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:38:51.400381 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:38:51.400382 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:38:51.400448 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:38:51.400759 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:38:51.400760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:38:51.400760 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:38:51.400930 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:38:51.400997 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 16:38:51.402212 (+ 1215us) server_negotiation.cc:299] Negotiation successful
0812 16:38:51.402311 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 16:38:51.403184 22193 heartbeater.cc:347] Connected to a master server at 127.0.253.254:42711
I20260812 16:38:51.403288 22193 heartbeater.cc:464] Registering TS with master...
I20260812 16:38:51.403489 22193 heartbeater.cc:511] Master 127.0.253.254:42711 requested a full tablet report, sending...
I20260812 16:38:51.403906 21496 ts_manager.cc:194] Registered new tserver with Master: 7aed80c2ca324bf09fae723cad2eb10b (127.0.253.194:43395)
I20260812 16:38:51.404413 21496 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.194@KRBTEST.COM'} at 127.0.253.194:49541
WARNING: no policy specified for kudu/127.0.253.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab.
Entry for principal kudu/127.0.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab.
I20260812 16:38:51.434396 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.253.212:39765/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.195:0
--local_ip_for_outbound_sockets=127.0.253.195
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:42711
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:34767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:38:51.569036 22207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:38:51.569408 22207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:38:51.569496 22207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:38:51.571317 22207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:38:51.571558 22207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:38:51.571658 22207 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.195
I20260812 16:38:51.573907 22207 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:34767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.0.253.212:39765/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:42711
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.195
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:38:51.574379 22207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:38:51.574713 22207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:38:51.582239 22221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:38:51.582528 22217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:38:51.582821 22207 server_base.cc:1034] running on GCE node
W20260812 16:38:51.583593 22219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:38:51.584203 22207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:38:51.584486 22207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:38:51.585633 22207 hybrid_clock.cc:648] HybridClock initialized: now 1786552731585612 us; error 35 us; skew 500 ppm
Aug 12 16:38:51 dist-test-slave-w4v5 krb5kdc[20592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552731, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:38:51.589022 22207 init.cc:377] Logged in from keytab as kudu/127.0.253.195@KRBTEST.COM (short username kudu)
I20260812 16:38:51.589538 22207 webserver.cc:460] Webserver started at http://127.0.253.195:33417/ using document root <none> and password file <none>
I20260812 16:38:51.589814 22207 fs_manager.cc:359] Metadata directory not provided
I20260812 16:38:51.589864 22207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:38:51.589986 22207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 16:38:51 dist-test-slave-w4v5 krb5kdc[20592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552731, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for HTTP/127.0.253.212@KRBTEST.COM
I20260812 16:38:51.612910 22207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-2/data/instance:
uuid: "0d2570a4a3444e39a273898e4b8e3d65"
format_stamp: "Formatted at 2026-08-12 16:38:51 on dist-test-slave-w4v5"
server_key: "478dad36d4c3477c4017639434565f13"
server_key_iv: "bb2fcdf7b36594359538ea29ab6a0a91"
server_key_version: "kuduclusterkey@0"
I20260812 16:38:51.613487 22207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance:
uuid: "0d2570a4a3444e39a273898e4b8e3d65"
format_stamp: "Formatted at 2026-08-12 16:38:51 on dist-test-slave-w4v5"
server_key: "478dad36d4c3477c4017639434565f13"
server_key_iv: "bb2fcdf7b36594359538ea29ab6a0a91"
server_key_version: "kuduclusterkey@0"
I20260812 16:38:51.617998 22207 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.000s
I20260812 16:38:51.639233 22234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:38:51.643010 22207 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.001s sys 0.000s
I20260812 16:38:51.643254 22207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-2/wal
uuid: "0d2570a4a3444e39a273898e4b8e3d65"
format_stamp: "Formatted at 2026-08-12 16:38:51 on dist-test-slave-w4v5"
server_key: "478dad36d4c3477c4017639434565f13"
server_key_iv: "bb2fcdf7b36594359538ea29ab6a0a91"
server_key_version: "kuduclusterkey@0"
I20260812 16:38:51.643466 22207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:38:51.663276 22207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:38:51.665632 22207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:38:51.665778 22207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:38:51.666044 22207 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:38:51.666427 22207 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:38:51.666472 22207 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:38:51.666501 22207 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:38:51.666519 22207 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:38:51.674235 22207 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.195:38191
I20260812 16:38:51.674680 22207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
I20260812 16:38:51.675380 22438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.195:38191 every 8 connection(s)
I20260812 16:38:51.679541 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 22207
I20260812 16:38:51.679641 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance
I20260812 16:38:51.679816 1015 external_mini_cluster.cc:1428] Setting key 6da7871cfee96d566a3d49be1e7c7539
Aug 12 16:38:51 dist-test-slave-w4v5 krb5kdc[20592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552731, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:38:51.687744 21913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:51.676345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:59251 (local address 127.0.253.254:42711)
0812 16:38:51.676397 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 16:38:51.676399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:38:51.676732 (+ 333us) server_negotiation.cc:407] Connection header received
0812 16:38:51.677426 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:38:51.677428 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:38:51.677440 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:38:51.677470 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:38:51.678286 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:51.678993 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:51.681279 (+ 2286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:51.681551 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:51.681593 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:38:51.684477 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:38:51.684479 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:38:51.684479 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:38:51.684487 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:38:51.686096 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:38:51.686582 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:38:51.686583 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:38:51.686584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:38:51.686651 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:38:51.686932 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:38:51.686933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:38:51.686933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:38:51.687103 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:38:51.687163 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 16:38:51.687570 (+ 407us) server_negotiation.cc:299] Negotiation successful
0812 16:38:51.687656 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 16:38:51.688279 22448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:51.676002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:42711 (local address 127.0.253.195:59251)
0812 16:38:51.676601 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0812 16:38:51.676647 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 16:38:51.677299 (+ 652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:38:51.677654 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:38:51.677658 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:38:51.677898 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:38:51.678179 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:38:51.678184 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:38:51.679174 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:38:51.679175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:38:51.681174 (+ 1999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:38:51.681177 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:38:51.681685 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:38:51.681686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:38:51.681823 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:38:51.682422 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:38:51.682433 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:38:51.684366 (+ 1933us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:38:51.686201 (+ 1835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:38:51.686208 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:38:51.686210 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:38:51.686492 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:38:51.686722 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:38:51.686723 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:38:51.686723 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:38:51.686854 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:38:51.687217 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:38:51.687224 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:38:51.687413 (+ 189us) client_negotiation.cc:769] Sending connection context
0812 16:38:51.687876 (+ 463us) client_negotiation.cc:240] Negotiation successful
0812 16:38:51.688035 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":81,"threads_started":1}
I20260812 16:38:51.688707 22440 heartbeater.cc:347] Connected to a master server at 127.0.253.254:42711
I20260812 16:38:51.688795 22440 heartbeater.cc:464] Registering TS with master...
I20260812 16:38:51.688978 22440 heartbeater.cc:511] Master 127.0.253.254:42711 requested a full tablet report, sending...
I20260812 16:38:51.689334 21496 ts_manager.cc:194] Registered new tserver with Master: 0d2570a4a3444e39a273898e4b8e3d65 (127.0.253.195:38191)
I20260812 16:38:51.689774 21496 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.195@KRBTEST.COM'} at 127.0.253.195:59251
I20260812 16:38:51.691380 1015 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:38:51.705121 21913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:51.695756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45296 (local address 127.0.253.254:42711)
0812 16:38:51.696354 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0812 16:38:51.696356 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:38:51.696365 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:38:51.696385 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:38:51.696386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:38:51.696396 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:38:51.696443 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:38:51.697238 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:51.697991 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:51.699726 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:51.700007 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:51.700053 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:38:51.701709 (+ 1656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:38:51.701710 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:38:51.701711 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:38:51.701718 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:38:51.703788 (+ 2070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:38:51.704184 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:38:51.704185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:38:51.704186 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:38:51.704244 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:38:51.704432 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:38:51.704432 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:38:51.704433 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:38:51.704554 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:38:51.704698 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 16:38:51.704938 (+ 240us) server_negotiation.cc:299] Negotiation successful
0812 16:38:51.705021 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573}
I20260812 16:38:51.712819 21496 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45296:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:38:51.715499 21496 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:38:51.725741 22474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:51.719330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:43395 (local address 127.0.0.1:37484)
0812 16:38:51.720256 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0812 16:38:51.720300 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 16:38:51.720367 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:38:51.720978 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:38:51.720982 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:38:51.720985 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:38:51.721071 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:38:51.721073 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:38:51.722731 (+ 1658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:38:51.722732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:38:51.724398 (+ 1666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:38:51.724401 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:38:51.725424 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:38:51.725426 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:38:51.725519 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:38:51.725539 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 16:38:51.725592 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 16:38:51.725660 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":893,"thread_start_us":87,"threads_started":1}
I20260812 16:38:51.726450 22467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:51.719443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37484 (local address 127.0.253.194:43395)
0812 16:38:51.720769 (+ 1326us) server_negotiation.cc:206] Beginning negotiation
0812 16:38:51.720781 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 16:38:51.720791 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:38:51.720833 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:38:51.720835 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:38:51.720852 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:38:51.721155 (+ 303us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:38:51.721235 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:51.722607 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:51.724520 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:51.725318 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:51.726206 (+ 888us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:38:51.726302 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 16:38:51.726329 (+ 27us) server_negotiation.cc:299] Negotiation successful
0812 16:38:51.726370 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1289,"thread_start_us":77,"threads_started":1}
I20260812 16:38:51.727731 22037 tablet_service.cc:1467] Processing CreateTablet for tablet 77b27edc1a6b4706ac6b5761aa0fd789 (DEFAULT_TABLE table=test-table [id=207c5c2f224445f499984c5112a0e5d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:38:51.728166 22037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77b27edc1a6b4706ac6b5761aa0fd789. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:38:51.732545 22478 tablet_bootstrap.cc:492] T 77b27edc1a6b4706ac6b5761aa0fd789 P 7aed80c2ca324bf09fae723cad2eb10b: Bootstrap starting.
I20260812 16:38:51.733654 22478 tablet_bootstrap.cc:654] T 77b27edc1a6b4706ac6b5761aa0fd789 P 7aed80c2ca324bf09fae723cad2eb10b: Neither blocks nor log segments found. Creating new log.
I20260812 16:38:51.734030 22478 log.cc:824] T 77b27edc1a6b4706ac6b5761aa0fd789 P 7aed80c2ca324bf09fae723cad2eb10b: Log is configured to *not* fsync() on all Append() calls
I20260812 16:38:51.734929 22478 tablet_bootstrap.cc:492] T 77b27edc1a6b4706ac6b5761aa0fd789 P 7aed80c2ca324bf09fae723cad2eb10b: No bootstrap required, opened a new log
I20260812 16:38:51.735011 22478 ts_tablet_manager.cc:1397] T 77b27edc1a6b4706ac6b5761aa0fd789 P 7aed80c2ca324bf09fae723cad2eb10b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 16:38:51.737458 22478 raft_consensus.cc:348] T 77b27edc1a6b4706ac6b5761aa0fd789 P 7aed80c2ca324bf09fae723cad2eb10b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aed80c2ca324bf09fae723cad2eb10b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 43395 } } peers { permanent_uuid: "0d2570a4a3444e39a273898e4b8e3d65" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38191 } } peers { permanent_uuid: "89de05aa958645f4b058bb40b1bc794d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 42673 } }
I20260812 16:38:51.737668 22478 raft_consensus.cc:374] T 77b27edc1a6b4706ac6b5761aa0fd789 P 7aed80c2ca324bf09fae723cad2eb10b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:38:51.737717 22478 raft_consensus.cc:729] T 77b27edc1a6b4706ac6b5761aa0fd789 P 7aed80c2ca324bf09fae723cad2eb10b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aed80c2ca324bf09fae723cad2eb10b, State: Initialized, Role: FOLLOWER
I20260812 16:38:51.737838 22478 consensus_queue.cc:260] T 77b27edc1a6b4706ac6b5761aa0fd789 P 7aed80c2ca324bf09fae723cad2eb10b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aed80c2ca324bf09fae723cad2eb10b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 43395 } } peers { permanent_uuid: "0d2570a4a3444e39a273898e4b8e3d65" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38191 } } peers { permanent_uuid: "89de05aa958645f4b058bb40b1bc794d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 42673 } }
I20260812 16:38:51.738121 22478 ts_tablet_manager.cc:1428] T 77b27edc1a6b4706ac6b5761aa0fd789 P 7aed80c2ca324bf09fae723cad2eb10b: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 16:38:51.738293 22193 heartbeater.cc:503] Master 127.0.253.254:42711 was elected leader, sending a full tablet report...
I20260812 16:38:51.741065 22463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:51.719794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.195:38191 (local address 127.0.0.1:60756)
0812 16:38:51.720592 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0812 16:38:51.720600 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 16:38:51.720649 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:38:51.727817 (+ 7168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:38:51.727818 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:38:51.727821 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:38:51.727870 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:38:51.727871 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:38:51.730868 (+ 2997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:38:51.730869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:38:51.732040 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:38:51.732044 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:38:51.740818 (+ 8774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:38:51.740819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:38:51.740892 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:38:51.740900 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 16:38:51.740932 (+ 32us) client_negotiation.cc:240] Negotiation successful
0812 16:38:51.740979 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":785,"thread_start_us":68,"threads_started":1}
I20260812 16:38:51.741675 22470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:51.719584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:42673 (local address 127.0.0.1:48792)
0812 16:38:51.720471 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0812 16:38:51.720479 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 16:38:51.720521 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:38:51.727738 (+ 7217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:38:51.727740 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:38:51.727742 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:38:51.727794 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:38:51.727795 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:38:51.739184 (+ 11389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:38:51.739186 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:38:51.740376 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:38:51.740381 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:38:51.741437 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:38:51.741438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:38:51.741506 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:38:51.741511 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:38:51.741544 (+ 33us) client_negotiation.cc:240] Negotiation successful
0812 16:38:51.741601 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":872,"thread_start_us":77,"threads_started":1}
I20260812 16:38:51.741712 22477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:51.719901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60756 (local address 127.0.253.195:38191)
0812 16:38:51.727453 (+ 7552us) server_negotiation.cc:206] Beginning negotiation
0812 16:38:51.727458 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:38:51.727467 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:38:51.727503 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:38:51.727505 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:38:51.727520 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:38:51.727601 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:38:51.729136 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:51.730282 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:51.739862 (+ 9580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:51.740708 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:51.741456 (+ 748us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:38:51.741555 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 16:38:51.741588 (+ 33us) server_negotiation.cc:299] Negotiation successful
0812 16:38:51.741632 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7509,"thread_start_us":86,"threads_started":1}
I20260812 16:38:51.742102 22472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:51.719704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48792 (local address 127.0.253.193:42673)
0812 16:38:51.727178 (+ 7474us) server_negotiation.cc:206] Beginning negotiation
0812 16:38:51.727184 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:38:51.727199 (+ 15us) server_negotiation.cc:407] Connection header received
0812 16:38:51.727240 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:38:51.727242 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:38:51.727260 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:38:51.727880 (+ 620us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:38:51.727953 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:51.729026 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:51.740505 (+ 11479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:51.741323 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:51.741870 (+ 547us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:38:51.741950 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 16:38:51.741978 (+ 28us) server_negotiation.cc:299] Negotiation successful
0812 16:38:51.742025 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7430,"thread_start_us":54,"threads_started":1}
I20260812 16:38:51.742697 21745 tablet_service.cc:1467] Processing CreateTablet for tablet 77b27edc1a6b4706ac6b5761aa0fd789 (DEFAULT_TABLE table=test-table [id=207c5c2f224445f499984c5112a0e5d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:38:51.742697 22301 tablet_service.cc:1467] Processing CreateTablet for tablet 77b27edc1a6b4706ac6b5761aa0fd789 (DEFAULT_TABLE table=test-table [id=207c5c2f224445f499984c5112a0e5d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:38:51.743068 21745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77b27edc1a6b4706ac6b5761aa0fd789. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:38:51.743115 22301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77b27edc1a6b4706ac6b5761aa0fd789. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:38:51.747136 22484 tablet_bootstrap.cc:492] T 77b27edc1a6b4706ac6b5761aa0fd789 P 89de05aa958645f4b058bb40b1bc794d: Bootstrap starting.
I20260812 16:38:51.747929 22484 tablet_bootstrap.cc:654] T 77b27edc1a6b4706ac6b5761aa0fd789 P 89de05aa958645f4b058bb40b1bc794d: Neither blocks nor log segments found. Creating new log.
I20260812 16:38:51.748288 22484 log.cc:824] T 77b27edc1a6b4706ac6b5761aa0fd789 P 89de05aa958645f4b058bb40b1bc794d: Log is configured to *not* fsync() on all Append() calls
I20260812 16:38:51.748668 22486 tablet_bootstrap.cc:492] T 77b27edc1a6b4706ac6b5761aa0fd789 P 0d2570a4a3444e39a273898e4b8e3d65: Bootstrap starting.
I20260812 16:38:51.749784 22486 tablet_bootstrap.cc:654] T 77b27edc1a6b4706ac6b5761aa0fd789 P 0d2570a4a3444e39a273898e4b8e3d65: Neither blocks nor log segments found. Creating new log.
I20260812 16:38:51.750123 22486 log.cc:824] T 77b27edc1a6b4706ac6b5761aa0fd789 P 0d2570a4a3444e39a273898e4b8e3d65: Log is configured to *not* fsync() on all Append() calls
I20260812 16:38:51.751017 22484 tablet_bootstrap.cc:492] T 77b27edc1a6b4706ac6b5761aa0fd789 P 89de05aa958645f4b058bb40b1bc794d: No bootstrap required, opened a new log
I20260812 16:38:51.751124 22484 ts_tablet_manager.cc:1397] T 77b27edc1a6b4706ac6b5761aa0fd789 P 89de05aa958645f4b058bb40b1bc794d: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260812 16:38:51.754029 22484 raft_consensus.cc:348] T 77b27edc1a6b4706ac6b5761aa0fd789 P 89de05aa958645f4b058bb40b1bc794d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aed80c2ca324bf09fae723cad2eb10b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 43395 } } peers { permanent_uuid: "0d2570a4a3444e39a273898e4b8e3d65" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38191 } } peers { permanent_uuid: "89de05aa958645f4b058bb40b1bc794d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 42673 } }
I20260812 16:38:51.754230 22484 raft_consensus.cc:374] T 77b27edc1a6b4706ac6b5761aa0fd789 P 89de05aa958645f4b058bb40b1bc794d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:38:51.754282 22484 raft_consensus.cc:729] T 77b27edc1a6b4706ac6b5761aa0fd789 P 89de05aa958645f4b058bb40b1bc794d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89de05aa958645f4b058bb40b1bc794d, State: Initialized, Role: FOLLOWER
I20260812 16:38:51.754448 22484 consensus_queue.cc:260] T 77b27edc1a6b4706ac6b5761aa0fd789 P 89de05aa958645f4b058bb40b1bc794d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aed80c2ca324bf09fae723cad2eb10b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 43395 } } peers { permanent_uuid: "0d2570a4a3444e39a273898e4b8e3d65" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38191 } } peers { permanent_uuid: "89de05aa958645f4b058bb40b1bc794d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 42673 } }
I20260812 16:38:51.754765 22484 ts_tablet_manager.cc:1428] T 77b27edc1a6b4706ac6b5761aa0fd789 P 89de05aa958645f4b058bb40b1bc794d: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260812 16:38:51.754904 21907 heartbeater.cc:503] Master 127.0.253.254:42711 was elected leader, sending a full tablet report...
I20260812 16:38:51.756366 22486 tablet_bootstrap.cc:492] T 77b27edc1a6b4706ac6b5761aa0fd789 P 0d2570a4a3444e39a273898e4b8e3d65: No bootstrap required, opened a new log
I20260812 16:38:51.756455 22486 ts_tablet_manager.cc:1397] T 77b27edc1a6b4706ac6b5761aa0fd789 P 0d2570a4a3444e39a273898e4b8e3d65: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260812 16:38:51.758765 22486 raft_consensus.cc:348] T 77b27edc1a6b4706ac6b5761aa0fd789 P 0d2570a4a3444e39a273898e4b8e3d65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aed80c2ca324bf09fae723cad2eb10b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 43395 } } peers { permanent_uuid: "0d2570a4a3444e39a273898e4b8e3d65" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38191 } } peers { permanent_uuid: "89de05aa958645f4b058bb40b1bc794d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 42673 } }
I20260812 16:38:51.758958 22486 raft_consensus.cc:374] T 77b27edc1a6b4706ac6b5761aa0fd789 P 0d2570a4a3444e39a273898e4b8e3d65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:38:51.759007 22486 raft_consensus.cc:729] T 77b27edc1a6b4706ac6b5761aa0fd789 P 0d2570a4a3444e39a273898e4b8e3d65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d2570a4a3444e39a273898e4b8e3d65, State: Initialized, Role: FOLLOWER
I20260812 16:38:51.759192 22486 consensus_queue.cc:260] T 77b27edc1a6b4706ac6b5761aa0fd789 P 0d2570a4a3444e39a273898e4b8e3d65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aed80c2ca324bf09fae723cad2eb10b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 43395 } } peers { permanent_uuid: "0d2570a4a3444e39a273898e4b8e3d65" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38191 } } peers { permanent_uuid: "89de05aa958645f4b058bb40b1bc794d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 42673 } }
I20260812 16:38:51.759524 22486 ts_tablet_manager.cc:1428] T 77b27edc1a6b4706ac6b5761aa0fd789 P 0d2570a4a3444e39a273898e4b8e3d65: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:38:51.759660 22440 heartbeater.cc:503] Master 127.0.253.254:42711 was elected leader, sending a full tablet report...
W20260812 16:38:51.874496 21909 tablet.cc:2369] T 77b27edc1a6b4706ac6b5761aa0fd789 P 89de05aa958645f4b058bb40b1bc794d: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 16:38:51 dist-test-slave-w4v5 krb5kdc[20592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552731, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for HTTP/127.0.253.212@KRBTEST.COM
W20260812 16:38:51.883229 22194 tablet.cc:2369] T 77b27edc1a6b4706ac6b5761aa0fd789 P 7aed80c2ca324bf09fae723cad2eb10b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 16:38:51.926801 22442 tablet.cc:2369] T 77b27edc1a6b4706ac6b5761aa0fd789 P 0d2570a4a3444e39a273898e4b8e3d65: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:38:51.997965 22494 raft_consensus.cc:482] T 77b27edc1a6b4706ac6b5761aa0fd789 P 0d2570a4a3444e39a273898e4b8e3d65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:38:51.998095 22494 raft_consensus.cc:504] T 77b27edc1a6b4706ac6b5761aa0fd789 P 0d2570a4a3444e39a273898e4b8e3d65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aed80c2ca324bf09fae723cad2eb10b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 43395 } } peers { permanent_uuid: "0d2570a4a3444e39a273898e4b8e3d65" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38191 } } peers { permanent_uuid: "89de05aa958645f4b058bb40b1bc794d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 42673 } }
I20260812 16:38:51.998528 22494 leader_election.cc:288] T 77b27edc1a6b4706ac6b5761aa0fd789 P 0d2570a4a3444e39a273898e4b8e3d65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aed80c2ca324bf09fae723cad2eb10b (127.0.253.194:43395), 89de05aa958645f4b058bb40b1bc794d (127.0.253.193:42673)
I20260812 16:38:52.002017 22448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:51.998728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:43395 (local address 127.0.253.195:33357)
0812 16:38:51.998802 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0812 16:38:51.998820 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 16:38:51.998898 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:38:51.999213 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:38:51.999214 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:38:51.999216 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:38:51.999301 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:38:51.999303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:38:52.000030 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:38:52.000031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:38:52.001030 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:38:52.001032 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:38:52.001755 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:38:52.001756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:38:52.001829 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:38:52.001833 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:38:52.001887 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 16:38:52.001930 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 16:38:52.002017 22467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:51.998864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:33357 (local address 127.0.253.194:43395)
0812 16:38:51.998995 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 16:38:51.998997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:38:51.999007 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:38:51.999040 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:38:51.999041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:38:51.999057 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:38:51.999136 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:38:51.999374 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:51.999939 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:52.001135 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:52.001657 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:52.001740 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:38:52.001750 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 16:38:52.001908 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 16:38:52.001947 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260812 16:38:52.002629 22472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:51.998975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:60537 (local address 127.0.253.193:42673)
0812 16:38:51.999022 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 16:38:51.999024 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:38:51.999114 (+ 90us) server_negotiation.cc:407] Connection header received
0812 16:38:51.999194 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:38:51.999195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:38:51.999205 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:38:51.999286 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:38:51.999477 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:52.000248 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:52.001650 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:52.002242 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:52.002333 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:38:52.002343 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 16:38:52.002518 (+ 175us) server_negotiation.cc:299] Negotiation successful
0812 16:38:52.002569 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 16:38:52.002648 22099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77b27edc1a6b4706ac6b5761aa0fd789" candidate_uuid: "0d2570a4a3444e39a273898e4b8e3d65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aed80c2ca324bf09fae723cad2eb10b" is_pre_election: true
I20260812 16:38:52.002816 22099 raft_consensus.cc:2454] T 77b27edc1a6b4706ac6b5761aa0fd789 P 7aed80c2ca324bf09fae723cad2eb10b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d2570a4a3444e39a273898e4b8e3d65 in term 0.
I20260812 16:38:52.002877 22496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:51.998860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:42673 (local address 127.0.253.195:60537)
0812 16:38:51.999027 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 16:38:51.999037 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:38:51.999131 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:38:51.999291 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:38:51.999291 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:38:51.999293 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:38:51.999381 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:38:51.999383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:38:52.000361 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:38:52.000362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:38:52.001550 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:38:52.001554 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:38:52.002346 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:38:52.002348 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:38:52.002424 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:38:52.002428 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:38:52.002734 (+ 306us) client_negotiation.cc:240] Negotiation successful
0812 16:38:52.002780 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":102,"threads_started":1}
I20260812 16:38:52.003086 22238 leader_election.cc:302] T 77b27edc1a6b4706ac6b5761aa0fd789 P 0d2570a4a3444e39a273898e4b8e3d65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d2570a4a3444e39a273898e4b8e3d65, 7aed80c2ca324bf09fae723cad2eb10b; no voters:
I20260812 16:38:52.003434 21811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77b27edc1a6b4706ac6b5761aa0fd789" candidate_uuid: "0d2570a4a3444e39a273898e4b8e3d65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89de05aa958645f4b058bb40b1bc794d" is_pre_election: true
I20260812 16:38:52.003575 21811 raft_consensus.cc:2454] T 77b27edc1a6b4706ac6b5761aa0fd789 P 89de05aa958645f4b058bb40b1bc794d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d2570a4a3444e39a273898e4b8e3d65 in term 0.
I20260812 16:38:52.003773 22494 raft_consensus.cc:2781] T 77b27edc1a6b4706ac6b5761aa0fd789 P 0d2570a4a3444e39a273898e4b8e3d65 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:38:52.003849 22494 raft_consensus.cc:482] T 77b27edc1a6b4706ac6b5761aa0fd789 P 0d2570a4a3444e39a273898e4b8e3d65 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:38:52.003887 22494 raft_consensus.cc:3037] T 77b27edc1a6b4706ac6b5761aa0fd789 P 0d2570a4a3444e39a273898e4b8e3d65 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:38:52.004843 22494 raft_consensus.cc:504] T 77b27edc1a6b4706ac6b5761aa0fd789 P 0d2570a4a3444e39a273898e4b8e3d65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aed80c2ca324bf09fae723cad2eb10b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 43395 } } peers { permanent_uuid: "0d2570a4a3444e39a273898e4b8e3d65" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38191 } } peers { permanent_uuid: "89de05aa958645f4b058bb40b1bc794d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 42673 } }
I20260812 16:38:52.004982 22494 leader_election.cc:288] T 77b27edc1a6b4706ac6b5761aa0fd789 P 0d2570a4a3444e39a273898e4b8e3d65 [CANDIDATE]: Term 1 election: Requested vote from peers 7aed80c2ca324bf09fae723cad2eb10b (127.0.253.194:43395), 89de05aa958645f4b058bb40b1bc794d (127.0.253.193:42673)
I20260812 16:38:52.005411 22099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77b27edc1a6b4706ac6b5761aa0fd789" candidate_uuid: "0d2570a4a3444e39a273898e4b8e3d65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aed80c2ca324bf09fae723cad2eb10b"
I20260812 16:38:52.005435 21811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77b27edc1a6b4706ac6b5761aa0fd789" candidate_uuid: "0d2570a4a3444e39a273898e4b8e3d65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89de05aa958645f4b058bb40b1bc794d"
I20260812 16:38:52.005498 22099 raft_consensus.cc:3037] T 77b27edc1a6b4706ac6b5761aa0fd789 P 7aed80c2ca324bf09fae723cad2eb10b [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:38:52.005522 21811 raft_consensus.cc:3037] T 77b27edc1a6b4706ac6b5761aa0fd789 P 89de05aa958645f4b058bb40b1bc794d [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:38:52.006384 22099 raft_consensus.cc:2454] T 77b27edc1a6b4706ac6b5761aa0fd789 P 7aed80c2ca324bf09fae723cad2eb10b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d2570a4a3444e39a273898e4b8e3d65 in term 1.
I20260812 16:38:52.006485 21811 raft_consensus.cc:2454] T 77b27edc1a6b4706ac6b5761aa0fd789 P 89de05aa958645f4b058bb40b1bc794d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d2570a4a3444e39a273898e4b8e3d65 in term 1.
I20260812 16:38:52.006723 22238 leader_election.cc:302] T 77b27edc1a6b4706ac6b5761aa0fd789 P 0d2570a4a3444e39a273898e4b8e3d65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d2570a4a3444e39a273898e4b8e3d65, 89de05aa958645f4b058bb40b1bc794d; no voters:
I20260812 16:38:52.006881 22494 raft_consensus.cc:2781] T 77b27edc1a6b4706ac6b5761aa0fd789 P 0d2570a4a3444e39a273898e4b8e3d65 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:38:52.007047 22494 raft_consensus.cc:686] T 77b27edc1a6b4706ac6b5761aa0fd789 P 0d2570a4a3444e39a273898e4b8e3d65 [term 1 LEADER]: Becoming Leader. State: Replica: 0d2570a4a3444e39a273898e4b8e3d65, State: Running, Role: LEADER
I20260812 16:38:52.007165 22494 consensus_queue.cc:237] T 77b27edc1a6b4706ac6b5761aa0fd789 P 0d2570a4a3444e39a273898e4b8e3d65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aed80c2ca324bf09fae723cad2eb10b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 43395 } } peers { permanent_uuid: "0d2570a4a3444e39a273898e4b8e3d65" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38191 } } peers { permanent_uuid: "89de05aa958645f4b058bb40b1bc794d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 42673 } }
I20260812 16:38:52.008057 21495 catalog_manager.cc:5495] T 77b27edc1a6b4706ac6b5761aa0fd789 P 0d2570a4a3444e39a273898e4b8e3d65 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d2570a4a3444e39a273898e4b8e3d65 (127.0.253.195). New cstate: current_term: 1 leader_uuid: "0d2570a4a3444e39a273898e4b8e3d65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aed80c2ca324bf09fae723cad2eb10b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 43395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d2570a4a3444e39a273898e4b8e3d65" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89de05aa958645f4b058bb40b1bc794d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 42673 } health_report { overall_health: UNKNOWN } } }
I20260812 16:38:52.077661 22477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:52.074273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60770 (local address 127.0.253.195:38191)
0812 16:38:52.074344 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 16:38:52.074346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:38:52.074355 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:38:52.074400 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:38:52.074401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:38:52.074422 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:38:52.074505 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:38:52.074696 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:52.075459 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:52.076573 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:38:52.076874 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:38:52.076940 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:38:52.077159 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:38:52.077302 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:38:52.077386 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 16:38:52.077560 (+ 174us) server_negotiation.cc:299] Negotiation successful
0812 16:38:52.077577 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 16:38:52.081928 21811 raft_consensus.cc:1261] T 77b27edc1a6b4706ac6b5761aa0fd789 P 89de05aa958645f4b058bb40b1bc794d [term 1 FOLLOWER]: Refusing update from remote peer 0d2570a4a3444e39a273898e4b8e3d65: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:38:52.081979 22099 raft_consensus.cc:1261] T 77b27edc1a6b4706ac6b5761aa0fd789 P 7aed80c2ca324bf09fae723cad2eb10b [term 1 FOLLOWER]: Refusing update from remote peer 0d2570a4a3444e39a273898e4b8e3d65: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:38:52.082401 22500 consensus_queue.cc:1035] T 77b27edc1a6b4706ac6b5761aa0fd789 P 0d2570a4a3444e39a273898e4b8e3d65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aed80c2ca324bf09fae723cad2eb10b" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 43395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:38:52.082587 22500 consensus_queue.cc:1035] T 77b27edc1a6b4706ac6b5761aa0fd789 P 0d2570a4a3444e39a273898e4b8e3d65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89de05aa958645f4b058bb40b1bc794d" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 42673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:38:52.086936 22506 mvcc.cc:205] Tried to move back new op lower bound from 7317719990602018816 to 7317719990301863936. Current Snapshot: MvccSnapshot[applied={T|T < 7317719990602018816}]
I20260812 16:38:52.087280 22505 mvcc.cc:205] Tried to move back new op lower bound from 7317719990602018816 to 7317719990301863936. Current Snapshot: MvccSnapshot[applied={T|T < 7317719990602018816}]
I20260812 16:38:52.091588 21495 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45296:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:38:52.091691 21495 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45296:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:38:52.092732 21495 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a0d9e0a42074ad5a6f0564378f355a9: Sending DeleteTablet for 3 replicas of tablet 77b27edc1a6b4706ac6b5761aa0fd789
I20260812 16:38:52.093309 22037 tablet_service.cc:1514] Processing DeleteTablet for tablet 77b27edc1a6b4706ac6b5761aa0fd789 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:38:52 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:37484
I20260812 16:38:52.093663 21745 tablet_service.cc:1514] Processing DeleteTablet for tablet 77b27edc1a6b4706ac6b5761aa0fd789 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:38:52 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:48792
I20260812 16:38:52.094010 22301 tablet_service.cc:1514] Processing DeleteTablet for tablet 77b27edc1a6b4706ac6b5761aa0fd789 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:38:52 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:60756
I20260812 16:38:52.094714 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 21654
I20260812 16:38:52.115239 22516 tablet_replica.cc:330] T 77b27edc1a6b4706ac6b5761aa0fd789 P 7aed80c2ca324bf09fae723cad2eb10b: stopping tablet replica
I20260812 16:38:52.115507 22516 raft_consensus.cc:2229] T 77b27edc1a6b4706ac6b5761aa0fd789 P 7aed80c2ca324bf09fae723cad2eb10b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:38:52.115772 22516 raft_consensus.cc:2258] T 77b27edc1a6b4706ac6b5761aa0fd789 P 7aed80c2ca324bf09fae723cad2eb10b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:38:52.116508 22516 ts_tablet_manager.cc:1905] T 77b27edc1a6b4706ac6b5761aa0fd789 P 7aed80c2ca324bf09fae723cad2eb10b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 16:38:52.117451 21464 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.253.193:42673 (error 108)
I20260812 16:38:52.118280 22518 tablet_replica.cc:330] T 77b27edc1a6b4706ac6b5761aa0fd789 P 0d2570a4a3444e39a273898e4b8e3d65: stopping tablet replica
I20260812 16:38:52.118475 22516 ts_tablet_manager.cc:1918] T 77b27edc1a6b4706ac6b5761aa0fd789 P 7aed80c2ca324bf09fae723cad2eb10b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:38:52.118521 22470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:52.118389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:42673 (local address 127.0.0.1:48798)
0812 16:38:52.118445 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 16:38:52.118484 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.193:42673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 16:38:52.118611 22516 log.cc:1196] T 77b27edc1a6b4706ac6b5761aa0fd789 P 7aed80c2ca324bf09fae723cad2eb10b: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-1/wal/wals/77b27edc1a6b4706ac6b5761aa0fd789
W20260812 16:38:52.118791 21464 catalog_manager.cc:4584] TS 89de05aa958645f4b058bb40b1bc794d (127.0.253.193:42673): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 77b27edc1a6b4706ac6b5761aa0fd789: Network error: Client connection negotiation failed: client connection to 127.0.253.193:42673: connect: Connection refused (error 111)
I20260812 16:38:52.118932 22516 ts_tablet_manager.cc:1939] T 77b27edc1a6b4706ac6b5761aa0fd789 P 7aed80c2ca324bf09fae723cad2eb10b: Deleting consensus metadata
I20260812 16:38:52.119426 21464 catalog_manager.cc:4841] TS 7aed80c2ca324bf09fae723cad2eb10b (127.0.253.194:43395): tablet 77b27edc1a6b4706ac6b5761aa0fd789 (table test-table [id=207c5c2f224445f499984c5112a0e5d5]) successfully deleted
I20260812 16:38:52.119647 22518 raft_consensus.cc:2229] T 77b27edc1a6b4706ac6b5761aa0fd789 P 0d2570a4a3444e39a273898e4b8e3d65 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:38:52.119809 22518 raft_consensus.cc:2258] T 77b27edc1a6b4706ac6b5761aa0fd789 P 0d2570a4a3444e39a273898e4b8e3d65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:38:52.119935 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 21927
I20260812 16:38:52.120335 22518 ts_tablet_manager.cc:1905] T 77b27edc1a6b4706ac6b5761aa0fd789 P 0d2570a4a3444e39a273898e4b8e3d65: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:38:52.126785 22518 ts_tablet_manager.cc:1918] T 77b27edc1a6b4706ac6b5761aa0fd789 P 0d2570a4a3444e39a273898e4b8e3d65: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:38:52.126919 22518 log.cc:1196] T 77b27edc1a6b4706ac6b5761aa0fd789 P 0d2570a4a3444e39a273898e4b8e3d65: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552619305671-1015-0/minicluster-data/ts-2/wal/wals/77b27edc1a6b4706ac6b5761aa0fd789
I20260812 16:38:52.127266 22518 ts_tablet_manager.cc:1939] T 77b27edc1a6b4706ac6b5761aa0fd789 P 0d2570a4a3444e39a273898e4b8e3d65: Deleting consensus metadata
I20260812 16:38:52.134825 21470 catalog_manager.cc:4841] TS 0d2570a4a3444e39a273898e4b8e3d65 (127.0.253.195:38191): tablet 77b27edc1a6b4706ac6b5761aa0fd789 (table test-table [id=207c5c2f224445f499984c5112a0e5d5]) successfully deleted
I20260812 16:38:52.135102 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 22207
I20260812 16:38:52.173416 22470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:52.172080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:42673 (local address 127.0.0.1:48802)
0812 16:38:52.173314 (+ 1234us) negotiation.cc:107] Waiting for socket to connect
0812 16:38:52.173365 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.193:42673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1200}
I20260812 16:38:52.174152 22470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:38:52.173749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:42673 (local address 127.0.0.1:48816)
0812 16:38:52.174088 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0812 16:38:52.174123 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.193:42673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320}
I20260812 16:38:52.180421 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 21427
I20260812 16:38:52.198803 1015 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 16:38:52.547992 1015 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 16:38:52.548105 1015 mini_ranger.cc:66] Stopping Ranger...
I20260812 16:38:52.893743 1015 mini_ranger.cc:68] Stopped Ranger
2026-08-12 16:38:52.893 UTC [20643] LOG: received smart shutdown request
2026-08-12 16:38:52.898 UTC [20643] LOG: background worker "logical replication launcher" (PID 20660) exited with exit code 1
2026-08-12 16:38:52.898 UTC [20652] LOG: shutting down
2026-08-12 16:38:53.084 UTC [20643] LOG: database system is shut down
2026-08-12T16:38:53Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (49842 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:38:53 dist-test-slave-w4v5 krb5kdc[22539](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:38:53 dist-test-slave-w4v5 krb5kdc[22539](info): set up 2 sockets
Aug 12 16:38:53 dist-test-slave-w4v5 krb5kdc[22539](info): commencing operation
krb5kdc: starting...
W20260812 16:38:54.508498 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.290s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:38:54 dist-test-slave-w4v5 krb5kdc[22539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:38:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:38:54Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.0.253.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.0.253.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.0.253.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/rangeradmin_127.0.253.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.0.253.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/rangeradmin_127.0.253.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.0.253.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.0.253.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.0.253.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/rangerlookup_127.0.253.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.0.253.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/rangerlookup_127.0.253.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.253.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.0.253.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.253.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/HTTP_127.0.253.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.253.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/HTTP_127.0.253.212@KRBTEST.COM.keytab.
I20260812 16:38:54.682711 1015 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskJXkJy5/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/postgres -l logfile start
2026-08-12 16:38:57.184 UTC [22606] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 16:38:57.184 UTC [22606] LOG: listening on IPv4 address "127.0.253.212", port 39535
2026-08-12 16:38:57.187 UTC [22606] LOG: listening on Unix socket "/tmp/.s.PGSQL.39535"
2026-08-12 16:38:57.210 UTC [22612] LOG: database system was shut down at 2026-08-12 16:38:55 UTC
2026-08-12 16:38:57.214 UTC [22606] LOG: database system is ready to accept connections
I20260812 16:38:58.430945 1015 mini_postgres.cc:96] Postgres bound to 39535
2026-08-12 16:38:58.438 UTC [22638] FATAL: database "slave" does not exist
127.0.253.212:39535 - accepting connections
I20260812 16:38:58.438987 1015 mini_ranger.cc:161] Starting Ranger...
I20260812 16:38:58.454730 1015 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 16:38:58.941100 1015 mini_ranger.cc:87] Created ranger Postgres database
I20260812 16:38:58.941191 1015 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ranger-admin
2026-08-12 16:38:59,232 [I] DB FLAVOR :POSTGRES
2026-08-12 16:38:59,232 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 16:38:59,232 [I] Checking connection..
2026-08-12 16:38:59,232 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:39535/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 16:38:59,513 [I] Checking connection passed.
2026-08-12 16:38:59,513 [I] --------- Verifying version history table ---------
2026-08-12 16:38:59,514 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:39535/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 16:38:59,759 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 16:38:59,760 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:39535/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 16:39:00,031 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 16:39:00,031 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 16:39:00,031 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:39535/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 16:39:00.299 UTC [22747] WARNING: there is no transaction in progress
2026-08-12 16:39:00,309 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 16:39:00,310 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:39535/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 16:39:00,580 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 16:39:00,581 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 16:39:00,581 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:39535/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 16:39:00,849 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:39535/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 16:39:01,089 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:39535/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-w4v5.c.gcp-upstream.internal','N') ;"
2026-08-12 16:39:01,357 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:39535/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 16:39:01,624 [I] Table x_portal_user does not exist in database ranger
2026-08-12 16:39:01,624 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:39535/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 16:39:01,902 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 16:39:01,902 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:39535/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 16:39:02,170 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:39535/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 16:39:02,419 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 16:39:02,419 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:39535/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 16:39:03.453 UTC [22959] WARNING: there is no transaction in progress
2026-08-12 16:39:03.471 UTC [22959] WARNING: there is no transaction in progress
2026-08-12 16:39:03.488 UTC [22959] WARNING: there is no transaction in progress
2026-08-12 16:39:03.503 UTC [22959] WARNING: there is no transaction in progress
2026-08-12 16:39:03.521 UTC [22959] WARNING: there is no transaction in progress
2026-08-12 16:39:03.540 UTC [22959] WARNING: there is no transaction in progress
2026-08-12 16:39:03.600 UTC [22959] WARNING: there is no transaction in progress
2026-08-12 16:39:03.608 UTC [22959] WARNING: there is no transaction in progress
2026-08-12 16:39:03.629 UTC [22959] WARNING: there is no transaction in progress
2026-08-12 16:39:03.640 UTC [22959] WARNING: there is no transaction in progress
2026-08-12 16:39:03.651 UTC [22959] WARNING: there is no transaction in progress
2026-08-12 16:39:03.666 UTC [22959] WARNING: there is no transaction in progress
2026-08-12 16:39:03.677 UTC [22959] WARNING: there is no transaction in progress
2026-08-12 16:39:03.684 UTC [22959] WARNING: there is no transaction in progress
2026-08-12 16:39:03.690 UTC [22959] WARNING: there is no transaction in progress
2026-08-12 16:39:03.698 UTC [22959] WARNING: there is no transaction in progress
2026-08-12 16:39:04,407 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 16:39:04,408 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:39535/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 16:39:04,678 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 16:39:04,678 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:39535/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 16:39:04,942 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 16:39:04,943 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:39535/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 16:39:05,211 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 16:39:05,211 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:39535/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 16:39:05,472 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:39535/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 16:39:05,724 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:39535/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-w4v5.c.gcp-upstream.internal';"
2026-08-12 16:39:05,990 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 16:39:05,991 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:39535/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 16:39:06,247 [I] DB_PATCHES have already been applied
I20260812 16:39:06.252894 1015 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ranger-admin:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskJXkJy5/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 16:39:06.252983 1015 mini_ranger.cc:193] Using host: 127.0.253.212
I20260812 16:39:06.257021 1015 mini_ranger.cc:227] Ranger admin URL: http://127.0.253.212:35959
Aug 12, 2026 4:39:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 4:39:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 4:39:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 4:39:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 4:39:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 4:39:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 4:39:07 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 16:39:07.690974 1015 mini_ranger.cc:160] Time spent starting Ranger: real 9.252s user 0.003s sys 0.008s
Aug 12, 2026 4:39:28 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 4:39:28 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 12, 2026 4:39:28 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 4:39:28 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 4:39:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 4:39:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 4:39:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 4:39:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 4:39:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 4:39:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 4:39:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 4:39:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 4:39:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 4:39:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 4:39:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 4:39:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 4:39:28 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 4:39:29 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 16:39:30.095917 1015 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.0.253.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.0.253.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.0.253.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/rangerkms_127.0.253.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.0.253.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/rangerkms_127.0.253.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.253.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.0.253.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.253.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/HTTP_127.0.253.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.253.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/HTTP_127.0.253.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 16:39:30 dist-test-slave-w4v5 krb5kdc[22539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552770, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 16:39:30.188720 1015 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 16:39:30.199846 1015 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 16:39:31.137943 1015 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 16:39:31.138041 1015 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ranger-kms
I20260812 16:39:31.138060 1015 mini_ranger_kms.cc:227] Using postgres at 127.0.253.212:39535
2026-08-12 16:39:31,313 [I] DB FLAVOR :POSTGRES
2026-08-12 16:39:31,313 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 16:39:31,314 [I] Checking connection
2026-08-12 16:39:31,314 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:39535/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 16:39:31,594 [I] connection success
2026-08-12 16:39:31,594 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 16:39:31,594 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 16:39:31,594 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:39535/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 16:39:31,843 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 16:39:31,844 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 16:39:31,844 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 16:39:31,844 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.253.212:39535/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 16:39:32,126 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 16:39:32.593570 1015 mini_ranger_kms.cc:310] Created kms service
I20260812 16:39:32.801978 1015 mini_ranger_kms.cc:326] Created kudu user
I20260812 16:39:32.848393 1015 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 16:39:32 dist-test-slave-w4v5 krb5kdc[22539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552770, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.0.253.212@KRBTEST.COM
I20260812 16:39:33.311888 1015 mini_ranger_kms.cc:384] Added ranger policy
I20260812 16:39:33.312116 1015 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ranger-kms:/tmp/dist-test-taskJXkJy5/build/release/bin/postgresql.jar:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskJXkJy5/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 16:39:33.312143 1015 mini_ranger_kms.cc:240] Using host: 127.0.253.212
I20260812 16:39:33.316517 1015 mini_ranger_kms.cc:276] Ranger KMS PID: 23277
I20260812 16:39:33.316599 1015 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.0.253.212:44131
Aug 12, 2026 4:39:34 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 4:39:34 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 4:39:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 4:39:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 4:39:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 4:39:34 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 16:39:35 dist-test-slave-w4v5 krb5kdc[22539](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786552775, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.253.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 12 16:39:36 dist-test-slave-w4v5 krb5kdc[22539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552775, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.253.212@KRBTEST.COM for HTTP/127.0.253.212@KRBTEST.COM
I20260812 16:39:36.241684 1015 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.053s user 0.004s sys 0.155s
I20260812 16:39:36.241783 1015 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 16:39:36.241830 1015 mini_ranger_kms.cc:401] 127.0.253.212:44131/kms/v1/keys
Aug 12 16:39:36 dist-test-slave-w4v5 krb5kdc[22539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552775, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.253.212@KRBTEST.COM for HTTP/127.0.253.212@KRBTEST.COM
Aug 12 16:39:38 dist-test-slave-w4v5 krb5kdc[22539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.0.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
I20260812 16:39:38.945629 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:43169
--webserver_interface=127.0.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:38711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:43169
--ranger_config_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.253.212:44131/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:39:39.030045 23369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:39:39.030264 23369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:39:39.030287 23369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:39:39.031600 23369 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:39:39.031649 23369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:39:39.031663 23369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:39:39.031677 23369 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:39:39.031689 23369 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:39:39.033339 23369 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:38711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/master-0/wal
--ranger_kms_url=127.0.253.212:44131/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:43169
--ranger_config_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:43169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:39:39.033588 23369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:39:39.033859 23369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:39:39.040259 23379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:39:39.040367 23383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:39:39.040405 23369 server_base.cc:1034] running on GCE node
W20260812 16:39:39.040266 23378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:39:39.040823 23369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:39:39.041188 23369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:39:39.042362 23369 hybrid_clock.cc:648] HybridClock initialized: now 1786552779042340 us; error 41 us; skew 500 ppm
Aug 12 16:39:39 dist-test-slave-w4v5 krb5kdc[22539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552779, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:39:39.049198 23369 init.cc:377] Logged in from keytab as kudu/127.0.253.254@KRBTEST.COM (short username kudu)
I20260812 16:39:39.049774 23369 webserver.cc:460] Webserver started at http://127.0.253.254:44911/ using document root <none> and password file <none>
I20260812 16:39:39.050053 23369 fs_manager.cc:359] Metadata directory not provided
I20260812 16:39:39.050125 23369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:39:39.050288 23369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 16:39:39 dist-test-slave-w4v5 krb5kdc[22539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552779, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for HTTP/127.0.253.212@KRBTEST.COM
I20260812 16:39:39.097985 23369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "c13e5efdd2d14293aa0c641db4caceec"
format_stamp: "Formatted at 2026-08-12 16:39:39 on dist-test-slave-w4v5"
server_key: "9761cb915b40924b1982fe05c07134c4"
server_key_iv: "16e3bcaf91adacef220b3f07bffce642"
server_key_version: "kuduclusterkey@0"
I20260812 16:39:39.098455 23369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "c13e5efdd2d14293aa0c641db4caceec"
format_stamp: "Formatted at 2026-08-12 16:39:39 on dist-test-slave-w4v5"
server_key: "9761cb915b40924b1982fe05c07134c4"
server_key_iv: "16e3bcaf91adacef220b3f07bffce642"
server_key_version: "kuduclusterkey@0"
I20260812 16:39:39.100171 23369 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:39:39.112578 23396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:39.112838 23369 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 16:39:39.112950 23369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "c13e5efdd2d14293aa0c641db4caceec"
format_stamp: "Formatted at 2026-08-12 16:39:39 on dist-test-slave-w4v5"
server_key: "9761cb915b40924b1982fe05c07134c4"
server_key_iv: "16e3bcaf91adacef220b3f07bffce642"
server_key_version: "kuduclusterkey@0"
I20260812 16:39:39.113082 23369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:39:39.122208 23369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:39:39.124377 23369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:39:39.124536 23369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:39:39.129647 23369 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:43169
I20260812 16:39:39.129743 23505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:43169 every 8 connection(s)
I20260812 16:39:39.130131 23369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:39:39.131007 23508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:39:39.132314 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 23369
I20260812 16:39:39.132457 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:39:39.132659 1015 external_mini_cluster.cc:1428] Setting key bd4be1bb716ab86133a8d42fea5b1eee
I20260812 16:39:39.134369 23508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec: Bootstrap starting.
I20260812 16:39:39.135121 23508 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec: Neither blocks nor log segments found. Creating new log.
I20260812 16:39:39.135449 23508 log.cc:824] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec: Log is configured to *not* fsync() on all Append() calls
I20260812 16:39:39.136257 23508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec: No bootstrap required, opened a new log
I20260812 16:39:39.138895 23508 raft_consensus.cc:348] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13e5efdd2d14293aa0c641db4caceec" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 43169 } }
I20260812 16:39:39.139104 23508 raft_consensus.cc:374] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:39:39.139178 23508 raft_consensus.cc:729] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c13e5efdd2d14293aa0c641db4caceec, State: Initialized, Role: FOLLOWER
I20260812 16:39:39.139338 23508 consensus_queue.cc:260] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13e5efdd2d14293aa0c641db4caceec" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 43169 } }
I20260812 16:39:39.139417 23508 raft_consensus.cc:388] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:39:39.139446 23508 raft_consensus.cc:482] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:39:39.139509 23508 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec [term 0 FOLLOWER]: Advancing to term 1
Aug 12 16:39:39 dist-test-slave-w4v5 krb5kdc[22539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:39:39.140156 23508 raft_consensus.cc:504] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13e5efdd2d14293aa0c641db4caceec" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 43169 } }
I20260812 16:39:39.140268 23508 leader_election.cc:302] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c13e5efdd2d14293aa0c641db4caceec; no voters:
I20260812 16:39:39.140429 23508 leader_election.cc:288] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:39:39.140472 23517 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:39:39.140734 23517 raft_consensus.cc:686] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec [term 1 LEADER]: Becoming Leader. State: Replica: c13e5efdd2d14293aa0c641db4caceec, State: Running, Role: LEADER
I20260812 16:39:39.140787 23508 sys_catalog.cc:564] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:39:39.140918 23517 consensus_queue.cc:237] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13e5efdd2d14293aa0c641db4caceec" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 43169 } }
I20260812 16:39:39.141265 23517 sys_catalog.cc:455] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec [sys.catalog]: SysCatalogTable state changed. Reason: New leader c13e5efdd2d14293aa0c641db4caceec. Latest consensus state: current_term: 1 leader_uuid: "c13e5efdd2d14293aa0c641db4caceec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13e5efdd2d14293aa0c641db4caceec" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 43169 } } }
I20260812 16:39:39.141273 23519 sys_catalog.cc:455] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c13e5efdd2d14293aa0c641db4caceec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13e5efdd2d14293aa0c641db4caceec" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 43169 } } }
I20260812 16:39:39.141427 23517 sys_catalog.cc:458] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec [sys.catalog]: This master's current role is: LEADER
I20260812 16:39:39.141539 23519 sys_catalog.cc:458] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec [sys.catalog]: This master's current role is: LEADER
I20260812 16:39:39.141705 23535 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:39:39.142237 23535 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:39:39.142611 23508 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 16:39:39.143893 23535 catalog_manager.cc:1347] Generated new cluster ID: 63a9582730f543e7be0c21db997b04b1
I20260812 16:39:39.143939 23535 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:39:39.146428 23513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:39.133196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55124 (local address 127.0.253.254:43169)
0812 16:39:39.133368 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:39.133379 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:39.133428 (+ 49us) server_negotiation.cc:407] Connection header received
0812 16:39:39.134046 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:39.134054 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:39.134122 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:39.134280 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:39:39.134982 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:39.136241 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:39.137024 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:39.137441 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:39.137563 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:39.140176 (+ 2613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:39:39.140180 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:39:39.140181 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:39:39.140193 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:39:39.144386 (+ 4193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:39.144835 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:39.144838 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:39.144838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:39.144934 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:39.145161 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:39.145161 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:39.145162 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:39.145455 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:39:39.146018 (+ 563us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:39.146092 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 16:39:39.146216 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":80,"threads_started":1}
I20260812 16:39:39.172169 23535 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:39:39.173035 23535 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:39:39.182886 23535 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec: Generated new TSK 0
I20260812 16:39:39.183188 23535 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab.
Entry for principal kudu/127.0.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab.
I20260812 16:39:39.954218 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.253.212:44131/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.193:0
--local_ip_for_outbound_sockets=127.0.253.193
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:43169
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:38711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:39:40.067801 23586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:39:40.068008 23586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:39:40.068032 23586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:39:40.069190 23586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:39:40.069235 23586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:39:40.069262 23586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.193
I20260812 16:39:40.070875 23586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:38711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.0.253.212:44131/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:43169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23586
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.193
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:39:40.071141 23586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:39:40.071403 23586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:39:40.077061 23595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:39:40.077066 23598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:39:40.077062 23592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:39:40.078058 23586 server_base.cc:1034] running on GCE node
I20260812 16:39:40.078306 23586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:39:40.078584 23586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:39:40.079769 23586 hybrid_clock.cc:648] HybridClock initialized: now 1786552780079735 us; error 32 us; skew 500 ppm
Aug 12 16:39:40 dist-test-slave-w4v5 krb5kdc[22539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552780, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:39:40.085664 23586 init.cc:377] Logged in from keytab as kudu/127.0.253.193@KRBTEST.COM (short username kudu)
I20260812 16:39:40.086252 23586 webserver.cc:460] Webserver started at http://127.0.253.193:43571/ using document root <none> and password file <none>
I20260812 16:39:40.086529 23586 fs_manager.cc:359] Metadata directory not provided
I20260812 16:39:40.086602 23586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:39:40.086755 23586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 16:39:40 dist-test-slave-w4v5 krb5kdc[22539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552780, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for HTTP/127.0.253.212@KRBTEST.COM
I20260812 16:39:40.100455 23586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-0/data/instance:
uuid: "2a33ab44b895425f8b5cd2a4957728a1"
format_stamp: "Formatted at 2026-08-12 16:39:40 on dist-test-slave-w4v5"
server_key: "9d506ff0da6a2086b06121eab25b40b9"
server_key_iv: "9f02957b1e43767068df005951394bd1"
server_key_version: "kuduclusterkey@0"
I20260812 16:39:40.100943 23586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance:
uuid: "2a33ab44b895425f8b5cd2a4957728a1"
format_stamp: "Formatted at 2026-08-12 16:39:40 on dist-test-slave-w4v5"
server_key: "9d506ff0da6a2086b06121eab25b40b9"
server_key_iv: "9f02957b1e43767068df005951394bd1"
server_key_version: "kuduclusterkey@0"
I20260812 16:39:40.102761 23586 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 16:39:40.120669 23612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:40.120935 23586 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260812 16:39:40.121042 23586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-0/wal
uuid: "2a33ab44b895425f8b5cd2a4957728a1"
format_stamp: "Formatted at 2026-08-12 16:39:40 on dist-test-slave-w4v5"
server_key: "9d506ff0da6a2086b06121eab25b40b9"
server_key_iv: "9f02957b1e43767068df005951394bd1"
server_key_version: "kuduclusterkey@0"
I20260812 16:39:40.121129 23586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:39:40.134392 23586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:39:40.136699 23586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:39:40.136874 23586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:39:40.137121 23586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:39:40.137526 23586 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:39:40.137583 23586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:40.137651 23586 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:39:40.137696 23586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:40.144609 23586 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.193:35007
I20260812 16:39:40.145145 23586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
I20260812 16:39:40.147212 23832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.193:35007 every 8 connection(s)
I20260812 16:39:40.147305 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 23586
I20260812 16:39:40.147399 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance
I20260812 16:39:40.147648 1015 external_mini_cluster.cc:1428] Setting key b77a45daf0400aac9a4b0bc098716a93
WARNING: no policy specified for kudu/127.0.253.194@KRBTEST.COM; defaulting to no policy
Aug 12 16:39:40 dist-test-slave-w4v5 krb5kdc[22539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552780, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.194@KRBTEST.COM" created.
I20260812 16:39:40.172955 23838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:40.148418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:50329 (local address 127.0.253.254:43169)
0812 16:39:40.150651 (+ 2233us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:40.150654 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:40.152107 (+ 1453us) server_negotiation.cc:407] Connection header received
0812 16:39:40.152137 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:40.152138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:40.152155 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:40.152222 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:39:40.153031 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:40.153845 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:40.155392 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:40.155718 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:40.155777 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:40.159752 (+ 3975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:39:40.159754 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:39:40.159754 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:39:40.159763 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:39:40.162191 (+ 2428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:40.171688 (+ 9497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:40.171690 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:40.171690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:40.171774 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:40.172085 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:40.172086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:40.172086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:40.172236 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:39:40.172292 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:40.172689 (+ 397us) server_negotiation.cc:299] Negotiation successful
0812 16:39:40.172840 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2199,"thread_start_us":85,"threads_started":1}
I20260812 16:39:40.173611 23837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:40.148129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:43169 (local address 127.0.253.193:50329)
0812 16:39:40.151239 (+ 3110us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:40.151296 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 16:39:40.151961 (+ 665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:39:40.152358 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:39:40.152362 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:39:40.152624 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:39:40.152920 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:40.152926 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:40.153958 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:40.153959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:40.155293 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:40.155297 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:40.155862 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:40.155862 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:40.156011 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:40.157971 (+ 1960us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:39:40.157983 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:39:40.159619 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:39:40.171143 (+ 11524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:40.171155 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:40.171159 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:40.171543 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:40.171851 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:40.171852 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:40.171853 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:40.172008 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:40.172327 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:39:40.172335 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:39:40.172512 (+ 177us) client_negotiation.cc:769] Sending connection context
0812 16:39:40.173181 (+ 669us) client_negotiation.cc:240] Negotiation successful
0812 16:39:40.173361 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3054,"thread_start_us":95,"threads_started":1}
I20260812 16:39:40.174305 23833 heartbeater.cc:347] Connected to a master server at 127.0.253.254:43169
I20260812 16:39:40.174412 23833 heartbeater.cc:464] Registering TS with master...
I20260812 16:39:40.174680 23833 heartbeater.cc:511] Master 127.0.253.254:43169 requested a full tablet report, sending...
I20260812 16:39:40.175403 23428 ts_manager.cc:194] Registered new tserver with Master: 2a33ab44b895425f8b5cd2a4957728a1 (127.0.253.193:35007)
I20260812 16:39:40.176220 23428 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.193@KRBTEST.COM'} at 127.0.253.193:50329
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab.
Entry for principal kudu/127.0.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab.
I20260812 16:39:40.185855 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.253.212:44131/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.194:0
--local_ip_for_outbound_sockets=127.0.253.194
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:43169
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:38711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 16:39:40 dist-test-slave-w4v5 krb5kdc[22539](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786552780, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 16:39:40.312937 23849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:39:40.313313 23849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:39:40.313409 23849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:39:40.315248 23849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:39:40.315410 23849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:39:40.315513 23849 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.194
I20260812 16:39:40.317956 23849 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:38711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.0.253.212:44131/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:43169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.194
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:39:40.318444 23849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:39:40.318830 23849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:39:40.324939 23862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:39:40.325031 23860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:39:40.324932 23864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:39:40.325608 23849 server_base.cc:1034] running on GCE node
I20260812 16:39:40.325815 23849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:39:40.326056 23849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:39:40.327222 23849 hybrid_clock.cc:648] HybridClock initialized: now 1786552780327202 us; error 33 us; skew 500 ppm
Aug 12 16:39:40 dist-test-slave-w4v5 krb5kdc[22539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552780, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:39:40.330034 23849 init.cc:377] Logged in from keytab as kudu/127.0.253.194@KRBTEST.COM (short username kudu)
I20260812 16:39:40.330550 23849 webserver.cc:460] Webserver started at http://127.0.253.194:38899/ using document root <none> and password file <none>
I20260812 16:39:40.330821 23849 fs_manager.cc:359] Metadata directory not provided
I20260812 16:39:40.330866 23849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:39:40.331025 23849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 16:39:40 dist-test-slave-w4v5 krb5kdc[22539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552780, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for HTTP/127.0.253.212@KRBTEST.COM
I20260812 16:39:40.348887 23849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-1/data/instance:
uuid: "f47490f7e2784f42b1b1330aab8ec5e7"
format_stamp: "Formatted at 2026-08-12 16:39:40 on dist-test-slave-w4v5"
server_key: "c4871399749b24c222e0ed1e6a62f52f"
server_key_iv: "867effcaa88d3ffa6ebc8989d8f39dc5"
server_key_version: "kuduclusterkey@0"
I20260812 16:39:40.349377 23849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance:
uuid: "f47490f7e2784f42b1b1330aab8ec5e7"
format_stamp: "Formatted at 2026-08-12 16:39:40 on dist-test-slave-w4v5"
server_key: "c4871399749b24c222e0ed1e6a62f52f"
server_key_iv: "867effcaa88d3ffa6ebc8989d8f39dc5"
server_key_version: "kuduclusterkey@0"
I20260812 16:39:40.351272 23849 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 16:39:40.384688 23873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:40.384994 23849 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.001s sys 0.000s
I20260812 16:39:40.385102 23849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-1/wal
uuid: "f47490f7e2784f42b1b1330aab8ec5e7"
format_stamp: "Formatted at 2026-08-12 16:39:40 on dist-test-slave-w4v5"
server_key: "c4871399749b24c222e0ed1e6a62f52f"
server_key_iv: "867effcaa88d3ffa6ebc8989d8f39dc5"
server_key_version: "kuduclusterkey@0"
I20260812 16:39:40.385222 23849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:39:40.416254 23849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:39:40.418922 23849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:39:40.419206 23849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:39:40.419562 23849 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:39:40.420047 23849 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:39:40.420151 23849 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:40.420331 23849 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:39:40.420408 23849 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:40.427832 23849 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.194:34169
I20260812 16:39:40.428262 23849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
I20260812 16:39:40.433694 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 23849
I20260812 16:39:40.433818 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance
I20260812 16:39:40.434018 1015 external_mini_cluster.cc:1428] Setting key eead39b35eb10ee808cac7344048df05
Aug 12 16:39:40 dist-test-slave-w4v5 krb5kdc[22539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552780, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:39:40.428855 24098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.194:34169 every 8 connection(s)
I20260812 16:39:40.440354 24101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:40.429083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:43169 (local address 127.0.253.194:55409)
0812 16:39:40.429268 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:40.429313 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 16:39:40.429983 (+ 670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:39:40.430374 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:39:40.430379 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:39:40.430667 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:39:40.430884 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:40.430889 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:40.431963 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:40.431964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:40.433265 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:40.433268 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:40.433839 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:40.433841 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:40.433990 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:40.434623 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:39:40.434731 (+ 108us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:39:40.436583 (+ 1852us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:39:40.438473 (+ 1890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:40.438481 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:40.438484 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:40.438818 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:40.439050 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:40.439051 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:40.439051 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:40.439232 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:40.439560 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:39:40.439568 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:39:40.439755 (+ 187us) client_negotiation.cc:769] Sending connection context
0812 16:39:40.439870 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 16:39:40.440044 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":113,"threads_started":1}
I20260812 16:39:40.440912 23838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:40.429407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:55409 (local address 127.0.253.254:43169)
0812 16:39:40.429471 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:40.429473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:40.429489 (+ 16us) server_negotiation.cc:407] Connection header received
0812 16:39:40.430126 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:40.430128 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:40.430147 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:40.430179 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:39:40.431025 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:40.431833 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:40.433374 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:40.433656 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:40.433721 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:40.436719 (+ 2998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:39:40.436721 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:39:40.436721 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:39:40.436730 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:39:40.438377 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:40.438913 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:40.438914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:40.438914 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:40.438979 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:40.439310 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:40.439311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:40.439311 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:40.439458 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:39:40.439525 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:40.440655 (+ 1130us) server_negotiation.cc:299] Negotiation successful
0812 16:39:40.440780 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 16:39:40.441362 24099 heartbeater.cc:347] Connected to a master server at 127.0.253.254:43169
I20260812 16:39:40.441498 24099 heartbeater.cc:464] Registering TS with master...
I20260812 16:39:40.441826 24099 heartbeater.cc:511] Master 127.0.253.254:43169 requested a full tablet report, sending...
I20260812 16:39:40.442344 23428 ts_manager.cc:194] Registered new tserver with Master: f47490f7e2784f42b1b1330aab8ec5e7 (127.0.253.194:34169)
I20260812 16:39:40.442921 23428 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.194@KRBTEST.COM'} at 127.0.253.194:55409
WARNING: no policy specified for kudu/127.0.253.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab.
Entry for principal kudu/127.0.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab.
I20260812 16:39:40.483307 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.253.212:44131/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.195:0
--local_ip_for_outbound_sockets=127.0.253.195
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:43169
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:38711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:39:40.587859 24106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:39:40.588111 24106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:39:40.588153 24106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:39:40.589514 24106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:39:40.589581 24106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:39:40.589625 24106 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.195
I20260812 16:39:40.591889 24106 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:38711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.0.253.212:44131/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:43169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24106
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.195
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:39:40.592276 24106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:39:40.592577 24106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:39:40.603977 24112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:39:40.604132 24117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:39:40.604004 24113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:39:40.605234 24106 server_base.cc:1034] running on GCE node
I20260812 16:39:40.605474 24106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:39:40.605758 24106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:39:40.606938 24106 hybrid_clock.cc:648] HybridClock initialized: now 1786552780606917 us; error 39 us; skew 500 ppm
Aug 12 16:39:40 dist-test-slave-w4v5 krb5kdc[22539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552780, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:39:40.610580 24106 init.cc:377] Logged in from keytab as kudu/127.0.253.195@KRBTEST.COM (short username kudu)
I20260812 16:39:40.611124 24106 webserver.cc:460] Webserver started at http://127.0.253.195:42433/ using document root <none> and password file <none>
I20260812 16:39:40.611403 24106 fs_manager.cc:359] Metadata directory not provided
I20260812 16:39:40.611454 24106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:39:40.611588 24106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 16:39:40 dist-test-slave-w4v5 krb5kdc[22539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552780, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for HTTP/127.0.253.212@KRBTEST.COM
I20260812 16:39:40.629047 24106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-2/data/instance:
uuid: "992b611ae5674367bdcfc1c966a3d914"
format_stamp: "Formatted at 2026-08-12 16:39:40 on dist-test-slave-w4v5"
server_key: "84629d7a179b819b2f2113d6ec90bfe6"
server_key_iv: "57ddd3b51ace7f1a21c923e13cdc5926"
server_key_version: "kuduclusterkey@0"
I20260812 16:39:40.630357 24106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance:
uuid: "992b611ae5674367bdcfc1c966a3d914"
format_stamp: "Formatted at 2026-08-12 16:39:40 on dist-test-slave-w4v5"
server_key: "84629d7a179b819b2f2113d6ec90bfe6"
server_key_iv: "57ddd3b51ace7f1a21c923e13cdc5926"
server_key_version: "kuduclusterkey@0"
I20260812 16:39:40.632565 24106 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260812 16:39:40.655344 24127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:40.655627 24106 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.000s sys 0.001s
I20260812 16:39:40.655747 24106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-2/wal
uuid: "992b611ae5674367bdcfc1c966a3d914"
format_stamp: "Formatted at 2026-08-12 16:39:40 on dist-test-slave-w4v5"
server_key: "84629d7a179b819b2f2113d6ec90bfe6"
server_key_iv: "57ddd3b51ace7f1a21c923e13cdc5926"
server_key_version: "kuduclusterkey@0"
I20260812 16:39:40.655850 24106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:39:40.672993 24106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:39:40.675586 24106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:39:40.675796 24106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:39:40.676110 24106 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:39:40.676534 24106 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:39:40.676628 24106 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:40.676704 24106 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:39:40.676766 24106 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:40.683625 24106 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.195:44745
I20260812 16:39:40.684295 24106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
I20260812 16:39:40.687252 24317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.195:44745 every 8 connection(s)
I20260812 16:39:40.691359 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 24106
I20260812 16:39:40.691480 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance
I20260812 16:39:40.691658 1015 external_mini_cluster.cc:1428] Setting key ae48b7503db1abb1050b39fcc6ba95cc
Aug 12 16:39:40 dist-test-slave-w4v5 krb5kdc[22539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552780, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:39:40.701368 23838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:40.688246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:57631 (local address 127.0.253.254:43169)
0812 16:39:40.688295 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:40.688298 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:40.690055 (+ 1757us) server_negotiation.cc:407] Connection header received
0812 16:39:40.690845 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:40.690847 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:40.690862 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:40.690909 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:39:40.691888 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:40.692709 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:40.694896 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:40.695227 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:40.695275 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:40.697984 (+ 2709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:39:40.697985 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:39:40.697986 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:39:40.697994 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:39:40.699783 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:40.700274 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:40.700276 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:40.700276 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:40.700334 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:40.700627 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:40.700628 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:40.700628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:40.700761 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:39:40.700802 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:40.701187 (+ 385us) server_negotiation.cc:299] Negotiation successful
0812 16:39:40.701274 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 16:39:40.702455 24326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:40.687944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:43169 (local address 127.0.253.195:57631)
0812 16:39:40.689906 (+ 1962us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:40.689955 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 16:39:40.690718 (+ 763us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:39:40.691159 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:39:40.691164 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:39:40.691440 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:39:40.691764 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:40.691772 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:40.693430 (+ 1658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:40.693431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:40.694788 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:40.694791 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:40.695381 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:40.695382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:40.695552 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:40.696175 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:39:40.696186 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:39:40.697871 (+ 1685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:39:40.699882 (+ 2011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:40.699889 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:40.699892 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:40.700185 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:40.700405 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:40.700406 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:40.700406 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:40.700551 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:40.700849 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:39:40.700855 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:39:40.701021 (+ 166us) client_negotiation.cc:769] Sending connection context
0812 16:39:40.701963 (+ 942us) client_negotiation.cc:240] Negotiation successful
0812 16:39:40.702213 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1900,"thread_start_us":84,"threads_started":1}
I20260812 16:39:40.703179 24318 heartbeater.cc:347] Connected to a master server at 127.0.253.254:43169
I20260812 16:39:40.703281 24318 heartbeater.cc:464] Registering TS with master...
I20260812 16:39:40.703492 24318 heartbeater.cc:511] Master 127.0.253.254:43169 requested a full tablet report, sending...
I20260812 16:39:40.704556 23428 ts_manager.cc:194] Registered new tserver with Master: 992b611ae5674367bdcfc1c966a3d914 (127.0.253.195:44745)
I20260812 16:39:40.705047 23428 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.195@KRBTEST.COM'} at 127.0.253.195:57631
I20260812 16:39:40.715114 1015 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:39:40.716012 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 23586
Aug 12 16:39:40 dist-test-slave-w4v5 krb5kdc[22539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552780, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for HTTP/127.0.253.212@KRBTEST.COM
I20260812 16:39:40.845265 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 23849
I20260812 16:39:40.866325 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 24106
I20260812 16:39:40.878665 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 23369
I20260812 16:39:40.897930 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:43169
--webserver_interface=127.0.253.254
--webserver_port=44911
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:38711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:43169
--ranger_config_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.253.212:44131/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.253.212:44131/kms with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:39:41.003186 24328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:39:41.003412 24328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:39:41.003436 24328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:39:41.004606 24328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:39:41.004649 24328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:39:41.004663 24328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:39:41.004676 24328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:39:41.004688 24328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:39:41.006309 24328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:38711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.253.212:44131/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:43169
--ranger_config_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:43169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=44911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:39:41.006585 24328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:39:41.006817 24328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:39:41.011875 24349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:39:41.011940 24343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:39:41.012071 24328 server_base.cc:1034] running on GCE node
W20260812 16:39:41.011878 24345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:39:41.012404 24328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:39:41.012732 24328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:39:41.013877 24328 hybrid_clock.cc:648] HybridClock initialized: now 1786552781013842 us; error 55 us; skew 500 ppm
Aug 12 16:39:41 dist-test-slave-w4v5 krb5kdc[22539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552781, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:39:41.016477 24328 init.cc:377] Logged in from keytab as kudu/127.0.253.254@KRBTEST.COM (short username kudu)
I20260812 16:39:41.016955 24328 webserver.cc:460] Webserver started at http://127.0.253.254:44911/ using document root <none> and password file <none>
I20260812 16:39:41.017207 24328 fs_manager.cc:359] Metadata directory not provided
I20260812 16:39:41.017289 24328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 16:39:41 dist-test-slave-w4v5 krb5kdc[22539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552781, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for HTTP/127.0.253.212@KRBTEST.COM
I20260812 16:39:41.101118 24328 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:39:41.102097 24367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:41.102372 24328 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:39:41.102485 24328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "c13e5efdd2d14293aa0c641db4caceec"
format_stamp: "Formatted at 2026-08-12 16:39:39 on dist-test-slave-w4v5"
server_key: "9761cb915b40924b1982fe05c07134c4"
server_key_iv: "16e3bcaf91adacef220b3f07bffce642"
server_key_version: "kuduclusterkey@0"
I20260812 16:39:41.102980 24328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:39:41.128767 24328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:39:41.131256 24328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:39:41.131428 24328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:39:41.135556 24328 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:43169
I20260812 16:39:41.135635 24470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:43169 every 8 connection(s)
I20260812 16:39:41.136025 24328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:39:41.137462 24471 sys_catalog.cc:263] Verifying existing consensus state
I20260812 16:39:41.138092 24471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec: Bootstrap starting.
I20260812 16:39:41.139309 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 24328
I20260812 16:39:41.139626 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.253.212:44131/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.193:35007
--local_ip_for_outbound_sockets=127.0.253.193
--tserver_master_addrs=127.0.253.254:43169
--webserver_port=43571
--webserver_interface=127.0.253.193
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:38711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.253.212:44131/kms with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:39:41.146826 24471 log.cc:824] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec: Log is configured to *not* fsync() on all Append() calls
I20260812 16:39:41.148672 24471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 16:39:41.148993 24471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec: Bootstrap complete.
I20260812 16:39:41.151595 24471 raft_consensus.cc:348] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13e5efdd2d14293aa0c641db4caceec" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 43169 } }
I20260812 16:39:41.151906 24471 raft_consensus.cc:729] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c13e5efdd2d14293aa0c641db4caceec, State: Initialized, Role: FOLLOWER
I20260812 16:39:41.152127 24471 consensus_queue.cc:260] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13e5efdd2d14293aa0c641db4caceec" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 43169 } }
I20260812 16:39:41.152245 24471 raft_consensus.cc:388] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:39:41.152303 24471 raft_consensus.cc:482] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:39:41.152354 24471 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec [term 1 FOLLOWER]: Advancing to term 2
I20260812 16:39:41.153381 24471 raft_consensus.cc:504] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13e5efdd2d14293aa0c641db4caceec" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 43169 } }
I20260812 16:39:41.153509 24471 leader_election.cc:302] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c13e5efdd2d14293aa0c641db4caceec; no voters:
I20260812 16:39:41.153739 24471 leader_election.cc:288] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 16:39:41.153776 24475 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec [term 2 FOLLOWER]: Leader election won for term 2
I20260812 16:39:41.153954 24475 raft_consensus.cc:686] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec [term 2 LEADER]: Becoming Leader. State: Replica: c13e5efdd2d14293aa0c641db4caceec, State: Running, Role: LEADER
I20260812 16:39:41.154080 24471 sys_catalog.cc:564] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:39:41.154120 24475 consensus_queue.cc:237] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13e5efdd2d14293aa0c641db4caceec" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 43169 } }
I20260812 16:39:41.154505 24477 sys_catalog.cc:455] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c13e5efdd2d14293aa0c641db4caceec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13e5efdd2d14293aa0c641db4caceec" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 43169 } } }
I20260812 16:39:41.154590 24477 sys_catalog.cc:458] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec [sys.catalog]: This master's current role is: LEADER
I20260812 16:39:41.154975 24478 sys_catalog.cc:455] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec [sys.catalog]: SysCatalogTable state changed. Reason: New leader c13e5efdd2d14293aa0c641db4caceec. Latest consensus state: current_term: 2 leader_uuid: "c13e5efdd2d14293aa0c641db4caceec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13e5efdd2d14293aa0c641db4caceec" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 43169 } } }
I20260812 16:39:41.155045 24478 sys_catalog.cc:458] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec [sys.catalog]: This master's current role is: LEADER
I20260812 16:39:41.155223 24486 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:39:41.155746 24486 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:39:41.155961 24486 catalog_manager.cc:1259] Loaded cluster ID: 63a9582730f543e7be0c21db997b04b1
I20260812 16:39:41.156047 24486 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:39:41.156114 24471 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 16:39:41.160211 24486 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:39:41.160766 24486 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec: Loaded TSK: 0
I20260812 16:39:41.161082 24486 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 16:39:41.256727 24473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:39:41.257019 24473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:39:41.257059 24473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:39:41.258772 24473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:39:41.258908 24473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:39:41.258994 24473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.193
I20260812 16:39:41.261267 24473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:38711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.253.212:44131/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.193:35007
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.253.193
--webserver_port=43571
--tserver_master_addrs=127.0.253.254:43169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.193
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:39:41.261703 24473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:39:41.262023 24473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:39:41.269449 24540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:39:41.269701 24535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:39:41.269853 24473 server_base.cc:1034] running on GCE node
W20260812 16:39:41.270110 24536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:39:41.270406 24473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:39:41.270742 24473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:39:41.271929 24473 hybrid_clock.cc:648] HybridClock initialized: now 1786552781271890 us; error 52 us; skew 500 ppm
Aug 12 16:39:41 dist-test-slave-w4v5 krb5kdc[22539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552781, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:39:41.275496 24473 init.cc:377] Logged in from keytab as kudu/127.0.253.193@KRBTEST.COM (short username kudu)
I20260812 16:39:41.276147 24473 webserver.cc:460] Webserver started at http://127.0.253.193:43571/ using document root <none> and password file <none>
I20260812 16:39:41.276434 24473 fs_manager.cc:359] Metadata directory not provided
I20260812 16:39:41.276543 24473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 16:39:41 dist-test-slave-w4v5 krb5kdc[22539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552781, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for HTTP/127.0.253.212@KRBTEST.COM
I20260812 16:39:41.306077 24473 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260812 16:39:41.307410 24558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:41.307606 24473 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:39:41.307715 24473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-0/wal
uuid: "2a33ab44b895425f8b5cd2a4957728a1"
format_stamp: "Formatted at 2026-08-12 16:39:40 on dist-test-slave-w4v5"
server_key: "9d506ff0da6a2086b06121eab25b40b9"
server_key_iv: "9f02957b1e43767068df005951394bd1"
server_key_version: "kuduclusterkey@0"
I20260812 16:39:41.308197 24473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:39:41.325611 24473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:39:41.328235 24473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:39:41.328470 24473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:39:41.328807 24473 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:39:41.329243 24473 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:39:41.329331 24473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:41.329397 24473 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:39:41.329445 24473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:41.335918 24473 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.193:35007
I20260812 16:39:41.336364 24473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
I20260812 16:39:41.337193 24760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.193:35007 every 8 connection(s)
I20260812 16:39:41.337828 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 24473
I20260812 16:39:41.338172 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.253.212:44131/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.194:34169
--local_ip_for_outbound_sockets=127.0.253.194
--tserver_master_addrs=127.0.253.254:43169
--webserver_port=38899
--webserver_interface=127.0.253.194
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:38711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.253.212:44131/kms with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 16:39:41 dist-test-slave-w4v5 krb5kdc[22539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552781, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:39:41.357991 24764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:41.337383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:43169 (local address 127.0.253.193:48175)
0812 16:39:41.337580 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:41.337623 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 16:39:41.338531 (+ 908us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:39:41.346772 (+ 8241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:39:41.346777 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:39:41.347020 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:39:41.347384 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:41.347393 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:41.349203 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:41.349204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:41.350472 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:41.350476 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:41.351346 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:41.351347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:41.351482 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:41.352112 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:39:41.352123 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:39:41.353891 (+ 1768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:39:41.356263 (+ 2372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:41.356269 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:41.356271 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:41.356555 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:41.356827 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:41.356827 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:41.356828 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:41.356949 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:41.357371 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:39:41.357377 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:39:41.357529 (+ 152us) client_negotiation.cc:769] Sending connection context
0812 16:39:41.357621 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 16:39:41.357769 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":102,"threads_started":1}
I20260812 16:39:41.359122 24770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:41.339354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:48175 (local address 127.0.253.254:43169)
0812 16:39:41.346208 (+ 6854us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:41.346221 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:41.346274 (+ 53us) server_negotiation.cc:407] Connection header received
0812 16:39:41.346447 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:41.346456 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:41.346527 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:41.346709 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:39:41.347549 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:41.349065 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:41.350603 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:41.351228 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:41.352836 (+ 1608us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:41.354011 (+ 1175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:39:41.354016 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:39:41.354017 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:39:41.354028 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:39:41.356168 (+ 2140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:41.356651 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:41.356654 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:41.356654 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:41.356757 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:41.357017 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:41.357018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:41.357018 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:41.357291 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:39:41.358708 (+ 1417us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:41.358793 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 16:39:41.358912 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6776,"thread_start_us":114,"threads_started":1}
I20260812 16:39:41.360070 24761 heartbeater.cc:347] Connected to a master server at 127.0.253.254:43169
I20260812 16:39:41.360175 24761 heartbeater.cc:464] Registering TS with master...
I20260812 16:39:41.360373 24761 heartbeater.cc:511] Master 127.0.253.254:43169 requested a full tablet report, sending...
W20260812 16:39:41.361322 24761 heartbeater.cc:644] Failed to heartbeat to 127.0.253.254:43169 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 16:39:41.361397 24761 heartbeater.cc:464] Registering TS with master...
I20260812 16:39:41.361461 24761 heartbeater.cc:511] Master 127.0.253.254:43169 requested a full tablet report, sending...
I20260812 16:39:41.361874 24761 heartbeater.cc:464] Registering TS with master...
I20260812 16:39:41.361951 24761 heartbeater.cc:511] Master 127.0.253.254:43169 requested a full tablet report, sending...
W20260812 16:39:41.362306 24761 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 16:39:41.469414 24771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:39:41.469696 24771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:39:41.469738 24771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:39:41.471477 24771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:39:41.471544 24771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:39:41.471586 24771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.194
I20260812 16:39:41.473762 24771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:38711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.253.212:44131/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.194:34169
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.253.194
--webserver_port=38899
--tserver_master_addrs=127.0.253.254:43169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.194
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:39:41.474036 24771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:39:41.474306 24771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:39:41.480093 24782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:39:41.480105 24780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:39:41.480304 24771 server_base.cc:1034] running on GCE node
W20260812 16:39:41.480430 24785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:39:41.480679 24771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:39:41.480928 24771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:39:41.482098 24771 hybrid_clock.cc:648] HybridClock initialized: now 1786552781482084 us; error 32 us; skew 500 ppm
Aug 12 16:39:41 dist-test-slave-w4v5 krb5kdc[22539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552781, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:39:41.484833 24771 init.cc:377] Logged in from keytab as kudu/127.0.253.194@KRBTEST.COM (short username kudu)
I20260812 16:39:41.485368 24771 webserver.cc:460] Webserver started at http://127.0.253.194:38899/ using document root <none> and password file <none>
I20260812 16:39:41.485646 24771 fs_manager.cc:359] Metadata directory not provided
I20260812 16:39:41.485718 24771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 16:39:41 dist-test-slave-w4v5 krb5kdc[22539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552781, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for HTTP/127.0.253.212@KRBTEST.COM
I20260812 16:39:41.506896 24771 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:39:41.508939 24805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:41.509337 24771 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260812 16:39:41.509462 24771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-1/wal
uuid: "f47490f7e2784f42b1b1330aab8ec5e7"
format_stamp: "Formatted at 2026-08-12 16:39:40 on dist-test-slave-w4v5"
server_key: "c4871399749b24c222e0ed1e6a62f52f"
server_key_iv: "867effcaa88d3ffa6ebc8989d8f39dc5"
server_key_version: "kuduclusterkey@0"
I20260812 16:39:41.509866 24771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:39:41.541730 24771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:39:41.544381 24771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:39:41.544634 24771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:39:41.544977 24771 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:39:41.545419 24771 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:39:41.545503 24771 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:41.545567 24771 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:39:41.545624 24771 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:41.553134 24771 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.194:34169
I20260812 16:39:41.553699 24771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
I20260812 16:39:41.558291 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 24771
I20260812 16:39:41.558621 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.253.212:44131/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.195:44745
--local_ip_for_outbound_sockets=127.0.253.195
--tserver_master_addrs=127.0.253.254:43169
--webserver_port=42433
--webserver_interface=127.0.253.195
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:38711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.253.212:44131/kms with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:39:41.559768 25007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.194:34169 every 8 connection(s)
Aug 12 16:39:41 dist-test-slave-w4v5 krb5kdc[22539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552781, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:39:41.567031 24770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:41.554537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:39873 (local address 127.0.253.254:43169)
0812 16:39:41.554610 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:41.554613 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:41.554870 (+ 257us) server_negotiation.cc:407] Connection header received
0812 16:39:41.555574 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:41.555576 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:41.555593 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:41.555662 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:39:41.556310 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:41.556824 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:41.557853 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:41.558101 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:41.558179 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:41.562877 (+ 4698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:39:41.562879 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:39:41.562879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:39:41.562888 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:39:41.564914 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:41.565463 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:41.565465 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:41.565465 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:41.565562 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:41.565871 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:41.565872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:41.565872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:41.566116 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:39:41.566259 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:41.566819 (+ 560us) server_negotiation.cc:299] Negotiation successful
0812 16:39:41.566928 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 16:39:41.567937 25020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:41.554566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:43169 (local address 127.0.253.194:39873)
0812 16:39:41.554739 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:41.554778 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 16:39:41.555387 (+ 609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:39:41.555738 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:39:41.555742 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:39:41.555938 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:39:41.556201 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:41.556208 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:41.556931 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:41.556932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:41.557753 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:41.557755 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:41.558219 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:41.558220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:41.558372 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:41.560940 (+ 2568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:39:41.560951 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:39:41.562745 (+ 1794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:39:41.565030 (+ 2285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:41.565038 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:41.565041 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:41.565366 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:41.565640 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:41.565640 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:41.565641 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:41.565793 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:41.566410 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:39:41.566416 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:39:41.566641 (+ 225us) client_negotiation.cc:769] Sending connection context
0812 16:39:41.567481 (+ 840us) client_negotiation.cc:240] Negotiation successful
0812 16:39:41.567680 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":99,"threads_started":1}
I20260812 16:39:41.568708 25011 heartbeater.cc:347] Connected to a master server at 127.0.253.254:43169
I20260812 16:39:41.568867 25011 heartbeater.cc:464] Registering TS with master...
I20260812 16:39:41.569094 25011 heartbeater.cc:511] Master 127.0.253.254:43169 requested a full tablet report, sending...
W20260812 16:39:41.569766 25011 heartbeater.cc:644] Failed to heartbeat to 127.0.253.254:43169 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 16:39:41.569885 25011 heartbeater.cc:464] Registering TS with master...
I20260812 16:39:41.569988 25011 heartbeater.cc:511] Master 127.0.253.254:43169 requested a full tablet report, sending...
I20260812 16:39:41.570322 25011 heartbeater.cc:464] Registering TS with master...
I20260812 16:39:41.570436 25011 heartbeater.cc:511] Master 127.0.253.254:43169 requested a full tablet report, sending...
W20260812 16:39:41.570770 25011 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 16:39:41.669319 25023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:39:41.669610 25023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:39:41.669648 25023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:39:41.671375 25023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:39:41.671474 25023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:39:41.671540 25023 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.195
I20260812 16:39:41.673727 25023 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:38711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.253.212:44131/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.195:44745
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.253.195
--webserver_port=42433
--tserver_master_addrs=127.0.253.254:43169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.195
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:39:41.674158 25023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:39:41.674510 25023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:39:41.681660 25045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:39:41.682011 25036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:39:41.681660 25035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:39:41.682806 25023 server_base.cc:1034] running on GCE node
I20260812 16:39:41.683099 25023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:39:41.683393 25023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:39:41.684580 25023 hybrid_clock.cc:648] HybridClock initialized: now 1786552781684563 us; error 32 us; skew 500 ppm
Aug 12 16:39:41 dist-test-slave-w4v5 krb5kdc[22539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552781, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:39:41.687836 25023 init.cc:377] Logged in from keytab as kudu/127.0.253.195@KRBTEST.COM (short username kudu)
I20260812 16:39:41.688365 25023 webserver.cc:460] Webserver started at http://127.0.253.195:42433/ using document root <none> and password file <none>
I20260812 16:39:41.688604 25023 fs_manager.cc:359] Metadata directory not provided
I20260812 16:39:41.688655 25023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 16:39:41 dist-test-slave-w4v5 krb5kdc[22539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552781, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for HTTP/127.0.253.212@KRBTEST.COM
I20260812 16:39:41.703817 25023 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:39:41.704607 25062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:41.704767 25023 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:39:41.704861 25023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-2/wal
uuid: "992b611ae5674367bdcfc1c966a3d914"
format_stamp: "Formatted at 2026-08-12 16:39:40 on dist-test-slave-w4v5"
server_key: "84629d7a179b819b2f2113d6ec90bfe6"
server_key_iv: "57ddd3b51ace7f1a21c923e13cdc5926"
server_key_version: "kuduclusterkey@0"
I20260812 16:39:41.705200 25023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:39:41.719277 25023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:39:41.721606 25023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:39:41.721799 25023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:39:41.722043 25023 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:39:41.722369 25023 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:39:41.722426 25023 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:41.722470 25023 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:39:41.722498 25023 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:41.729542 25023 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.195:44745
I20260812 16:39:41.729979 25023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
I20260812 16:39:41.733976 25272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.195:44745 every 8 connection(s)
I20260812 16:39:41.738336 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 25023
Aug 12 16:39:41 dist-test-slave-w4v5 krb5kdc[22539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552781, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:39:41.745812 25279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:41.734777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:43169 (local address 127.0.253.195:38369)
0812 16:39:41.735138 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:41.735186 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 16:39:41.735817 (+ 631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:39:41.736237 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:39:41.736242 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:39:41.736495 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:39:41.736779 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:41.736786 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:41.737783 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:41.737784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:41.739042 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:41.739045 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:41.739655 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:41.739657 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:41.739811 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:41.740443 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:39:41.740455 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:39:41.741989 (+ 1534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:39:41.744037 (+ 2048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:41.744043 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:41.744046 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:41.744358 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:41.744609 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:41.744609 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:41.744610 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:41.744749 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:41.745063 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:39:41.745069 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:39:41.745234 (+ 165us) client_negotiation.cc:769] Sending connection context
0812 16:39:41.745377 (+ 143us) client_negotiation.cc:240] Negotiation successful
0812 16:39:41.745560 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":111,"threads_started":1}
I20260812 16:39:41.746555 24770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:41.734760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:38369 (local address 127.0.253.254:43169)
0812 16:39:41.735065 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:41.735068 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:41.735277 (+ 209us) server_negotiation.cc:407] Connection header received
0812 16:39:41.735956 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:41.735958 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:41.735973 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:41.736007 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:39:41.736898 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:41.737680 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:41.739172 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:41.739474 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:41.739565 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:41.742107 (+ 2542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:39:41.742109 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:39:41.742110 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:39:41.742117 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:39:41.743939 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:41.744463 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:41.744464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:41.744465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:41.744537 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:41.744832 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:41.744834 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:41.744834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:41.744970 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:39:41.745033 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:41.746361 (+ 1328us) server_negotiation.cc:299] Negotiation successful
0812 16:39:41.746451 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267}
I20260812 16:39:41.747665 25276 heartbeater.cc:347] Connected to a master server at 127.0.253.254:43169
I20260812 16:39:41.747803 25276 heartbeater.cc:464] Registering TS with master...
I20260812 16:39:41.748035 25276 heartbeater.cc:511] Master 127.0.253.254:43169 requested a full tablet report, sending...
W20260812 16:39:41.748831 25276 heartbeater.cc:644] Failed to heartbeat to 127.0.253.254:43169 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 16:39:41.748921 25276 heartbeater.cc:464] Registering TS with master...
I20260812 16:39:41.748996 25276 heartbeater.cc:511] Master 127.0.253.254:43169 requested a full tablet report, sending...
I20260812 16:39:41.749298 25276 heartbeater.cc:464] Registering TS with master...
I20260812 16:39:41.749374 25276 heartbeater.cc:511] Master 127.0.253.254:43169 requested a full tablet report, sending...
W20260812 16:39:41.749691 25276 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 16:39:41.752292 25280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:41.738753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55140 (local address 127.0.253.254:43169)
0812 16:39:41.739397 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:41.739399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:41.739406 (+ 7us) server_negotiation.cc:407] Connection header received
0812 16:39:41.739427 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:41.739427 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:41.739438 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:41.739497 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:39:41.740657 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:41.741375 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:41.747627 (+ 6252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:41.747941 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:41.748023 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:41.749477 (+ 1454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:39:41.749478 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:39:41.749479 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:39:41.749486 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:39:41.751176 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:41.751571 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:41.751572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:41.751572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:41.751624 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:41.751837 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:41.751838 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:41.751838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:41.751934 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:39:41.752050 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:41.752160 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 16:39:41.752226 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":90,"threads_started":1}
Aug 12 16:39:42 dist-test-slave-w4v5 krb5kdc[22539](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786552782, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:39:42.362514 24761 heartbeater.cc:464] Registering TS with master...
I20260812 16:39:42.362677 24761 heartbeater.cc:511] Master 127.0.253.254:43169 requested a full tablet report, sending...
I20260812 16:39:42.363178 24398 ts_manager.cc:194] Registered new tserver with Master: 2a33ab44b895425f8b5cd2a4957728a1 (127.0.253.193:35007)
I20260812 16:39:42.364188 24398 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.193@KRBTEST.COM'} at 127.0.253.193:48175
I20260812 16:39:42.571014 25011 heartbeater.cc:464] Registering TS with master...
I20260812 16:39:42.571223 25011 heartbeater.cc:511] Master 127.0.253.254:43169 requested a full tablet report, sending...
I20260812 16:39:42.571640 24398 ts_manager.cc:194] Registered new tserver with Master: f47490f7e2784f42b1b1330aab8ec5e7 (127.0.253.194:34169)
I20260812 16:39:42.572209 24398 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.194@KRBTEST.COM'} at 127.0.253.194:39873
Aug 12 16:39:42 dist-test-slave-w4v5 krb5kdc[22539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552782, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for HTTP/127.0.253.212@KRBTEST.COM
I20260812 16:39:42.749967 25276 heartbeater.cc:464] Registering TS with master...
I20260812 16:39:42.750154 25276 heartbeater.cc:511] Master 127.0.253.254:43169 requested a full tablet report, sending...
I20260812 16:39:42.750612 24398 ts_manager.cc:194] Registered new tserver with Master: 992b611ae5674367bdcfc1c966a3d914 (127.0.253.195:44745)
I20260812 16:39:42.751187 24398 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.195@KRBTEST.COM'} at 127.0.253.195:38369
I20260812 16:39:42.753468 1015 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:39:42.763576 25315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:42.754593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55148 (local address 127.0.253.254:43169)
0812 16:39:42.754890 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:42.754892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:42.754900 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:39:42.754924 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:42.754925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:42.754938 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:42.755022 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:39:42.755765 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:42.756601 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:42.757829 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:42.758137 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:42.759827 (+ 1690us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:42.759878 (+ 51us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:39:42.759879 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:39:42.759879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:39:42.759884 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:39:42.761777 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:42.762352 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:42.762353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:42.762353 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:42.762421 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:42.762649 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:42.762650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:42.762650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:42.762785 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:39:42.763345 (+ 560us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:42.763389 (+ 44us) server_negotiation.cc:299] Negotiation successful
0812 16:39:42.763475 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":66,"threads_started":1}
I20260812 16:39:42.767755 24398 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55148:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:39:42.770483 24398 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:39:42.780777 25334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:42.774444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.195:44745 (local address 127.0.0.1:60886)
0812 16:39:42.775290 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:42.775300 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:39:42.775357 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:39:42.775667 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:39:42.775671 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:39:42.775673 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:39:42.775746 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:42.775749 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:42.777386 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:42.777387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:42.778729 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:42.778732 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:42.780471 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:42.780472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:42.780557 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:42.780571 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 16:39:42.780638 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 16:39:42.780691 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":828,"thread_start_us":75,"threads_started":1}
I20260812 16:39:42.781095 25335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:42.774559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60886 (local address 127.0.253.195:44745)
0812 16:39:42.775225 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:42.775231 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:42.775415 (+ 184us) server_negotiation.cc:407] Connection header received
0812 16:39:42.775457 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:42.775460 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:42.775478 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:42.775554 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:39:42.775841 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:42.777279 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:42.779504 (+ 2225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:42.780297 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:42.780373 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:42.780433 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:42.780971 (+ 538us) server_negotiation.cc:299] Negotiation successful
0812 16:39:42.781017 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":58,"threads_started":1}
I20260812 16:39:42.782089 25125 tablet_service.cc:1467] Processing CreateTablet for tablet 7b017ed58d4847428c59ff3d73c55537 (DEFAULT_TABLE table=test-table [id=69cd702c5802434c8ed2a1d16739ee24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:39:42.782498 25125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b017ed58d4847428c59ff3d73c55537. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:39:42.787006 25331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:42.774264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:34169 (local address 127.0.0.1:33788)
0812 16:39:42.774801 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:42.774812 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:39:42.774865 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:39:42.779438 (+ 4573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:39:42.779439 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:39:42.779441 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:39:42.779496 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:42.779497 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:42.782518 (+ 3021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:42.782519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:42.783687 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:42.783690 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:42.786778 (+ 3088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:42.786779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:42.786854 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:42.786860 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 16:39:42.786889 (+ 29us) client_negotiation.cc:240] Negotiation successful
0812 16:39:42.786929 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":81,"threads_started":1}
I20260812 16:39:42.787451 25336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:42.774654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33788 (local address 127.0.253.194:34169)
0812 16:39:42.779201 (+ 4547us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:42.779206 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:42.779227 (+ 21us) server_negotiation.cc:407] Connection header received
0812 16:39:42.779267 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:42.779269 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:42.779286 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:42.779372 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:39:42.779680 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:42.780781 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:42.783794 (+ 3013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:42.784554 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:42.787220 (+ 2666us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:42.787299 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:42.787327 (+ 28us) server_negotiation.cc:299] Negotiation successful
0812 16:39:42.787373 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4506,"thread_start_us":60,"threads_started":1}
I20260812 16:39:42.787966 25333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:42.774043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:35007 (local address 127.0.0.1:53828)
0812 16:39:42.774441 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:42.774486 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 16:39:42.774862 (+ 376us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:39:42.779582 (+ 4720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:39:42.779582 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:39:42.779584 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:39:42.779611 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:42.779612 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:42.784672 (+ 5060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:42.784673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:42.785779 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:42.785781 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:42.787752 (+ 1971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:42.787753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:42.787827 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:42.787831 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:39:42.787860 (+ 29us) client_negotiation.cc:240] Negotiation successful
0812 16:39:42.787896 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":87,"threads_started":1}
I20260812 16:39:42.788121 24892 tablet_service.cc:1467] Processing CreateTablet for tablet 7b017ed58d4847428c59ff3d73c55537 (DEFAULT_TABLE table=test-table [id=69cd702c5802434c8ed2a1d16739ee24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:39:42.788486 24892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b017ed58d4847428c59ff3d73c55537. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:39:42.788519 25332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:42.774143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53828 (local address 127.0.253.193:35007)
0812 16:39:42.779268 (+ 5125us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:42.779273 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:42.779285 (+ 12us) server_negotiation.cc:407] Connection header received
0812 16:39:42.779324 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:42.779326 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:42.779343 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:42.781048 (+ 1705us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:39:42.781149 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:42.782404 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:42.785883 (+ 3479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:42.786665 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:42.788290 (+ 1625us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:42.788371 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:42.788397 (+ 26us) server_negotiation.cc:299] Negotiation successful
0812 16:39:42.788442 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5082,"thread_start_us":84,"threads_started":1}
I20260812 16:39:42.789199 24643 tablet_service.cc:1467] Processing CreateTablet for tablet 7b017ed58d4847428c59ff3d73c55537 (DEFAULT_TABLE table=test-table [id=69cd702c5802434c8ed2a1d16739ee24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:39:42.789530 24643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b017ed58d4847428c59ff3d73c55537. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:39:42.791226 25337 tablet_bootstrap.cc:492] T 7b017ed58d4847428c59ff3d73c55537 P 992b611ae5674367bdcfc1c966a3d914: Bootstrap starting.
I20260812 16:39:42.792032 25337 tablet_bootstrap.cc:654] T 7b017ed58d4847428c59ff3d73c55537 P 992b611ae5674367bdcfc1c966a3d914: Neither blocks nor log segments found. Creating new log.
I20260812 16:39:42.792785 25338 tablet_bootstrap.cc:492] T 7b017ed58d4847428c59ff3d73c55537 P f47490f7e2784f42b1b1330aab8ec5e7: Bootstrap starting.
I20260812 16:39:42.793365 25341 tablet_bootstrap.cc:492] T 7b017ed58d4847428c59ff3d73c55537 P 2a33ab44b895425f8b5cd2a4957728a1: Bootstrap starting.
I20260812 16:39:42.793493 25338 tablet_bootstrap.cc:654] T 7b017ed58d4847428c59ff3d73c55537 P f47490f7e2784f42b1b1330aab8ec5e7: Neither blocks nor log segments found. Creating new log.
I20260812 16:39:42.793891 25338 log.cc:824] T 7b017ed58d4847428c59ff3d73c55537 P f47490f7e2784f42b1b1330aab8ec5e7: Log is configured to *not* fsync() on all Append() calls
I20260812 16:39:42.794042 25341 tablet_bootstrap.cc:654] T 7b017ed58d4847428c59ff3d73c55537 P 2a33ab44b895425f8b5cd2a4957728a1: Neither blocks nor log segments found. Creating new log.
I20260812 16:39:42.794431 25341 log.cc:824] T 7b017ed58d4847428c59ff3d73c55537 P 2a33ab44b895425f8b5cd2a4957728a1: Log is configured to *not* fsync() on all Append() calls
I20260812 16:39:42.794716 25338 tablet_bootstrap.cc:492] T 7b017ed58d4847428c59ff3d73c55537 P f47490f7e2784f42b1b1330aab8ec5e7: No bootstrap required, opened a new log
I20260812 16:39:42.794796 25338 ts_tablet_manager.cc:1397] T 7b017ed58d4847428c59ff3d73c55537 P f47490f7e2784f42b1b1330aab8ec5e7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:39:42.795334 25341 tablet_bootstrap.cc:492] T 7b017ed58d4847428c59ff3d73c55537 P 2a33ab44b895425f8b5cd2a4957728a1: No bootstrap required, opened a new log
I20260812 16:39:42.795389 25341 ts_tablet_manager.cc:1397] T 7b017ed58d4847428c59ff3d73c55537 P 2a33ab44b895425f8b5cd2a4957728a1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:39:42.797083 25338 raft_consensus.cc:348] T 7b017ed58d4847428c59ff3d73c55537 P f47490f7e2784f42b1b1330aab8ec5e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a33ab44b895425f8b5cd2a4957728a1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 35007 } } peers { permanent_uuid: "992b611ae5674367bdcfc1c966a3d914" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 44745 } } peers { permanent_uuid: "f47490f7e2784f42b1b1330aab8ec5e7" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 34169 } }
I20260812 16:39:42.797268 25338 raft_consensus.cc:374] T 7b017ed58d4847428c59ff3d73c55537 P f47490f7e2784f42b1b1330aab8ec5e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:39:42.797312 25338 raft_consensus.cc:729] T 7b017ed58d4847428c59ff3d73c55537 P f47490f7e2784f42b1b1330aab8ec5e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f47490f7e2784f42b1b1330aab8ec5e7, State: Initialized, Role: FOLLOWER
I20260812 16:39:42.797443 25338 consensus_queue.cc:260] T 7b017ed58d4847428c59ff3d73c55537 P f47490f7e2784f42b1b1330aab8ec5e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a33ab44b895425f8b5cd2a4957728a1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 35007 } } peers { permanent_uuid: "992b611ae5674367bdcfc1c966a3d914" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 44745 } } peers { permanent_uuid: "f47490f7e2784f42b1b1330aab8ec5e7" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 34169 } }
I20260812 16:39:42.797549 25337 log.cc:824] T 7b017ed58d4847428c59ff3d73c55537 P 992b611ae5674367bdcfc1c966a3d914: Log is configured to *not* fsync() on all Append() calls
I20260812 16:39:42.797485 25341 raft_consensus.cc:348] T 7b017ed58d4847428c59ff3d73c55537 P 2a33ab44b895425f8b5cd2a4957728a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a33ab44b895425f8b5cd2a4957728a1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 35007 } } peers { permanent_uuid: "992b611ae5674367bdcfc1c966a3d914" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 44745 } } peers { permanent_uuid: "f47490f7e2784f42b1b1330aab8ec5e7" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 34169 } }
I20260812 16:39:42.797636 25341 raft_consensus.cc:374] T 7b017ed58d4847428c59ff3d73c55537 P 2a33ab44b895425f8b5cd2a4957728a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:39:42.797667 25341 raft_consensus.cc:729] T 7b017ed58d4847428c59ff3d73c55537 P 2a33ab44b895425f8b5cd2a4957728a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a33ab44b895425f8b5cd2a4957728a1, State: Initialized, Role: FOLLOWER
I20260812 16:39:42.797716 25338 ts_tablet_manager.cc:1428] T 7b017ed58d4847428c59ff3d73c55537 P f47490f7e2784f42b1b1330aab8ec5e7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:39:42.797775 25341 consensus_queue.cc:260] T 7b017ed58d4847428c59ff3d73c55537 P 2a33ab44b895425f8b5cd2a4957728a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a33ab44b895425f8b5cd2a4957728a1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 35007 } } peers { permanent_uuid: "992b611ae5674367bdcfc1c966a3d914" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 44745 } } peers { permanent_uuid: "f47490f7e2784f42b1b1330aab8ec5e7" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 34169 } }
I20260812 16:39:42.798018 25341 ts_tablet_manager.cc:1428] T 7b017ed58d4847428c59ff3d73c55537 P 2a33ab44b895425f8b5cd2a4957728a1: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 16:39:42.798159 24761 heartbeater.cc:503] Master 127.0.253.254:43169 was elected leader, sending a full tablet report...
I20260812 16:39:42.798573 25011 heartbeater.cc:503] Master 127.0.253.254:43169 was elected leader, sending a full tablet report...
I20260812 16:39:42.801720 25337 tablet_bootstrap.cc:492] T 7b017ed58d4847428c59ff3d73c55537 P 992b611ae5674367bdcfc1c966a3d914: No bootstrap required, opened a new log
I20260812 16:39:42.801877 25337 ts_tablet_manager.cc:1397] T 7b017ed58d4847428c59ff3d73c55537 P 992b611ae5674367bdcfc1c966a3d914: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20260812 16:39:42.804620 25337 raft_consensus.cc:348] T 7b017ed58d4847428c59ff3d73c55537 P 992b611ae5674367bdcfc1c966a3d914 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a33ab44b895425f8b5cd2a4957728a1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 35007 } } peers { permanent_uuid: "992b611ae5674367bdcfc1c966a3d914" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 44745 } } peers { permanent_uuid: "f47490f7e2784f42b1b1330aab8ec5e7" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 34169 } }
I20260812 16:39:42.804884 25337 raft_consensus.cc:374] T 7b017ed58d4847428c59ff3d73c55537 P 992b611ae5674367bdcfc1c966a3d914 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:39:42.804973 25337 raft_consensus.cc:729] T 7b017ed58d4847428c59ff3d73c55537 P 992b611ae5674367bdcfc1c966a3d914 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 992b611ae5674367bdcfc1c966a3d914, State: Initialized, Role: FOLLOWER
I20260812 16:39:42.805179 25337 consensus_queue.cc:260] T 7b017ed58d4847428c59ff3d73c55537 P 992b611ae5674367bdcfc1c966a3d914 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a33ab44b895425f8b5cd2a4957728a1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 35007 } } peers { permanent_uuid: "992b611ae5674367bdcfc1c966a3d914" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 44745 } } peers { permanent_uuid: "f47490f7e2784f42b1b1330aab8ec5e7" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 34169 } }
I20260812 16:39:42.805541 25337 ts_tablet_manager.cc:1428] T 7b017ed58d4847428c59ff3d73c55537 P 992b611ae5674367bdcfc1c966a3d914: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260812 16:39:42.805719 25276 heartbeater.cc:503] Master 127.0.253.254:43169 was elected leader, sending a full tablet report...
W20260812 16:39:42.815614 25019 tablet.cc:2369] T 7b017ed58d4847428c59ff3d73c55537 P f47490f7e2784f42b1b1330aab8ec5e7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:39:42.822779 25356 raft_consensus.cc:482] T 7b017ed58d4847428c59ff3d73c55537 P 992b611ae5674367bdcfc1c966a3d914 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:39:42.822950 25356 raft_consensus.cc:504] T 7b017ed58d4847428c59ff3d73c55537 P 992b611ae5674367bdcfc1c966a3d914 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a33ab44b895425f8b5cd2a4957728a1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 35007 } } peers { permanent_uuid: "992b611ae5674367bdcfc1c966a3d914" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 44745 } } peers { permanent_uuid: "f47490f7e2784f42b1b1330aab8ec5e7" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 34169 } }
I20260812 16:39:42.823433 25356 leader_election.cc:288] T 7b017ed58d4847428c59ff3d73c55537 P 992b611ae5674367bdcfc1c966a3d914 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a33ab44b895425f8b5cd2a4957728a1 (127.0.253.193:35007), f47490f7e2784f42b1b1330aab8ec5e7 (127.0.253.194:34169)
I20260812 16:39:42.828186 25332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:42.823741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:38495 (local address 127.0.253.193:35007)
0812 16:39:42.823792 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:42.823795 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:42.823991 (+ 196us) server_negotiation.cc:407] Connection header received
0812 16:39:42.824112 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:42.824114 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:42.824126 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:42.824161 (+ 35us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:39:42.824424 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:42.825311 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:42.827021 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:42.827653 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:42.827696 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:42.827703 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:42.828049 (+ 346us) server_negotiation.cc:299] Negotiation successful
0812 16:39:42.828102 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 16:39:42.828471 25336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:42.824101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:45891 (local address 127.0.253.194:34169)
0812 16:39:42.824147 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:42.824149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:42.824260 (+ 111us) server_negotiation.cc:407] Connection header received
0812 16:39:42.824354 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:42.824356 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:42.824365 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:42.824396 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:39:42.824582 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:42.825362 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:42.827020 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:42.827635 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:42.827693 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:42.827703 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:42.828357 (+ 654us) server_negotiation.cc:299] Negotiation successful
0812 16:39:42.828412 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 16:39:42.828714 25357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:42.823666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:35007 (local address 127.0.253.195:38495)
0812 16:39:42.823916 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:42.823929 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:39:42.824025 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:39:42.824246 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:39:42.824247 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:39:42.824249 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:39:42.824341 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:42.824343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:42.825476 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:42.825476 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:42.826896 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:42.826900 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:42.827870 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:42.827871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:42.827956 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:42.827961 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:39:42.828525 (+ 564us) client_negotiation.cc:240] Negotiation successful
0812 16:39:42.828585 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":112,"thread_start_us":128,"threads_started":1}
I20260812 16:39:42.829166 24679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b017ed58d4847428c59ff3d73c55537" candidate_uuid: "992b611ae5674367bdcfc1c966a3d914" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a33ab44b895425f8b5cd2a4957728a1" is_pre_election: true
I20260812 16:39:42.829356 24679 raft_consensus.cc:2454] T 7b017ed58d4847428c59ff3d73c55537 P 2a33ab44b895425f8b5cd2a4957728a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992b611ae5674367bdcfc1c966a3d914 in term 0.
I20260812 16:39:42.829607 25358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:42.823915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:34169 (local address 127.0.253.195:45891)
0812 16:39:42.824205 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:42.824214 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 16:39:42.824280 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:39:42.824460 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:39:42.824460 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:39:42.824461 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:39:42.824514 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:42.824515 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:42.825468 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:42.825476 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:42.826896 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:42.826900 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:42.828168 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:42.828170 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:42.828253 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:42.828257 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:39:42.828701 (+ 444us) client_negotiation.cc:240] Negotiation successful
0812 16:39:42.828746 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":79,"thread_start_us":70,"threads_started":1}
I20260812 16:39:42.829650 25063 leader_election.cc:302] T 7b017ed58d4847428c59ff3d73c55537 P 992b611ae5674367bdcfc1c966a3d914 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a33ab44b895425f8b5cd2a4957728a1, 992b611ae5674367bdcfc1c966a3d914; no voters:
I20260812 16:39:42.829922 25356 raft_consensus.cc:2781] T 7b017ed58d4847428c59ff3d73c55537 P 992b611ae5674367bdcfc1c966a3d914 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:39:42.829988 25356 raft_consensus.cc:482] T 7b017ed58d4847428c59ff3d73c55537 P 992b611ae5674367bdcfc1c966a3d914 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:39:42.830013 25356 raft_consensus.cc:3037] T 7b017ed58d4847428c59ff3d73c55537 P 992b611ae5674367bdcfc1c966a3d914 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:39:42.830174 24933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b017ed58d4847428c59ff3d73c55537" candidate_uuid: "992b611ae5674367bdcfc1c966a3d914" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f47490f7e2784f42b1b1330aab8ec5e7" is_pre_election: true
I20260812 16:39:42.830340 24933 raft_consensus.cc:2454] T 7b017ed58d4847428c59ff3d73c55537 P f47490f7e2784f42b1b1330aab8ec5e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992b611ae5674367bdcfc1c966a3d914 in term 0.
I20260812 16:39:42.830945 25356 raft_consensus.cc:504] T 7b017ed58d4847428c59ff3d73c55537 P 992b611ae5674367bdcfc1c966a3d914 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a33ab44b895425f8b5cd2a4957728a1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 35007 } } peers { permanent_uuid: "992b611ae5674367bdcfc1c966a3d914" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 44745 } } peers { permanent_uuid: "f47490f7e2784f42b1b1330aab8ec5e7" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 34169 } }
I20260812 16:39:42.831117 25356 leader_election.cc:288] T 7b017ed58d4847428c59ff3d73c55537 P 992b611ae5674367bdcfc1c966a3d914 [CANDIDATE]: Term 1 election: Requested vote from peers 2a33ab44b895425f8b5cd2a4957728a1 (127.0.253.193:35007), f47490f7e2784f42b1b1330aab8ec5e7 (127.0.253.194:34169)
I20260812 16:39:42.831499 24933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b017ed58d4847428c59ff3d73c55537" candidate_uuid: "992b611ae5674367bdcfc1c966a3d914" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f47490f7e2784f42b1b1330aab8ec5e7"
I20260812 16:39:42.831585 24933 raft_consensus.cc:3037] T 7b017ed58d4847428c59ff3d73c55537 P f47490f7e2784f42b1b1330aab8ec5e7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:39:42.832681 24679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b017ed58d4847428c59ff3d73c55537" candidate_uuid: "992b611ae5674367bdcfc1c966a3d914" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a33ab44b895425f8b5cd2a4957728a1"
I20260812 16:39:42.832857 24679 raft_consensus.cc:3037] T 7b017ed58d4847428c59ff3d73c55537 P 2a33ab44b895425f8b5cd2a4957728a1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:39:42.833797 24933 raft_consensus.cc:2454] T 7b017ed58d4847428c59ff3d73c55537 P f47490f7e2784f42b1b1330aab8ec5e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992b611ae5674367bdcfc1c966a3d914 in term 1.
I20260812 16:39:42.833847 24679 raft_consensus.cc:2454] T 7b017ed58d4847428c59ff3d73c55537 P 2a33ab44b895425f8b5cd2a4957728a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992b611ae5674367bdcfc1c966a3d914 in term 1.
I20260812 16:39:42.833986 25063 leader_election.cc:302] T 7b017ed58d4847428c59ff3d73c55537 P 992b611ae5674367bdcfc1c966a3d914 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992b611ae5674367bdcfc1c966a3d914, f47490f7e2784f42b1b1330aab8ec5e7; no voters:
I20260812 16:39:42.834161 25356 raft_consensus.cc:2781] T 7b017ed58d4847428c59ff3d73c55537 P 992b611ae5674367bdcfc1c966a3d914 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:39:42.834331 25356 raft_consensus.cc:686] T 7b017ed58d4847428c59ff3d73c55537 P 992b611ae5674367bdcfc1c966a3d914 [term 1 LEADER]: Becoming Leader. State: Replica: 992b611ae5674367bdcfc1c966a3d914, State: Running, Role: LEADER
I20260812 16:39:42.834460 25356 consensus_queue.cc:237] T 7b017ed58d4847428c59ff3d73c55537 P 992b611ae5674367bdcfc1c966a3d914 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a33ab44b895425f8b5cd2a4957728a1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 35007 } } peers { permanent_uuid: "992b611ae5674367bdcfc1c966a3d914" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 44745 } } peers { permanent_uuid: "f47490f7e2784f42b1b1330aab8ec5e7" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 34169 } }
I20260812 16:39:42.835439 24395 catalog_manager.cc:5495] T 7b017ed58d4847428c59ff3d73c55537 P 992b611ae5674367bdcfc1c966a3d914 reported cstate change: term changed from 0 to 1, leader changed from <none> to 992b611ae5674367bdcfc1c966a3d914 (127.0.253.195). New cstate: current_term: 1 leader_uuid: "992b611ae5674367bdcfc1c966a3d914" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a33ab44b895425f8b5cd2a4957728a1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 35007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992b611ae5674367bdcfc1c966a3d914" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 44745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f47490f7e2784f42b1b1330aab8ec5e7" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 34169 } health_report { overall_health: UNKNOWN } } }
W20260812 16:39:42.837781 24762 tablet.cc:2369] T 7b017ed58d4847428c59ff3d73c55537 P 2a33ab44b895425f8b5cd2a4957728a1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:39:42.846305 25335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:42.842838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60894 (local address 127.0.253.195:44745)
0812 16:39:42.842908 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:42.842910 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:42.842918 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:39:42.842957 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:42.842958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:42.842979 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:42.843045 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:39:42.843254 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:42.843974 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:42.845086 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:42.845364 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:42.845436 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:42.845660 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:39:42.845799 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:39:42.845885 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:42.846196 (+ 311us) server_negotiation.cc:299] Negotiation successful
0812 16:39:42.846229 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 16:39:42.848606 24933 raft_consensus.cc:1261] T 7b017ed58d4847428c59ff3d73c55537 P f47490f7e2784f42b1b1330aab8ec5e7 [term 1 FOLLOWER]: Refusing update from remote peer 992b611ae5674367bdcfc1c966a3d914: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:39:42.849035 25359 consensus_queue.cc:1035] T 7b017ed58d4847428c59ff3d73c55537 P 992b611ae5674367bdcfc1c966a3d914 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f47490f7e2784f42b1b1330aab8ec5e7" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 34169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:39:42.851294 24679 raft_consensus.cc:1261] T 7b017ed58d4847428c59ff3d73c55537 P 2a33ab44b895425f8b5cd2a4957728a1 [term 1 FOLLOWER]: Refusing update from remote peer 992b611ae5674367bdcfc1c966a3d914: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:39:42.851620 25359 consensus_queue.cc:1035] T 7b017ed58d4847428c59ff3d73c55537 P 992b611ae5674367bdcfc1c966a3d914 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a33ab44b895425f8b5cd2a4957728a1" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 35007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:39:42.855628 25366 mvcc.cc:205] Tried to move back new op lower bound from 7317720198544748544 to 7317720198490681344. Current Snapshot: MvccSnapshot[applied={T|T < 7317720198544748544}]
I20260812 16:39:42.855844 25373 mvcc.cc:205] Tried to move back new op lower bound from 7317720198544748544 to 7317720198490681344. Current Snapshot: MvccSnapshot[applied={T|T < 7317720198544748544}]
I20260812 16:39:42.857764 25368 mvcc.cc:205] Tried to move back new op lower bound from 7317720198544748544 to 7317720198490681344. Current Snapshot: MvccSnapshot[applied={T|T < 7317720198544748544}]
I20260812 16:39:42.861820 24395 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:39:42.861943 24395 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:39:42.862696 24395 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c13e5efdd2d14293aa0c641db4caceec: Sending DeleteTablet for 3 replicas of tablet 7b017ed58d4847428c59ff3d73c55537
I20260812 16:39:42.863024 25125 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b017ed58d4847428c59ff3d73c55537 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:39:42 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:60886
I20260812 16:39:42.863152 24643 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b017ed58d4847428c59ff3d73c55537 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:39:42 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:53828
I20260812 16:39:42.863324 25384 tablet_replica.cc:330] T 7b017ed58d4847428c59ff3d73c55537 P 992b611ae5674367bdcfc1c966a3d914: stopping tablet replica
I20260812 16:39:42.863427 25384 raft_consensus.cc:2229] T 7b017ed58d4847428c59ff3d73c55537 P 992b611ae5674367bdcfc1c966a3d914 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:39:42.863441 24892 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b017ed58d4847428c59ff3d73c55537 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:39:42 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:33788
I20260812 16:39:42.863576 25384 raft_consensus.cc:2258] T 7b017ed58d4847428c59ff3d73c55537 P 992b611ae5674367bdcfc1c966a3d914 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:39:42.863675 25390 tablet_replica.cc:330] T 7b017ed58d4847428c59ff3d73c55537 P f47490f7e2784f42b1b1330aab8ec5e7: stopping tablet replica
I20260812 16:39:42.863853 25390 raft_consensus.cc:2229] T 7b017ed58d4847428c59ff3d73c55537 P f47490f7e2784f42b1b1330aab8ec5e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:39:42.863938 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 24473
I20260812 16:39:42.864020 25390 raft_consensus.cc:2258] T 7b017ed58d4847428c59ff3d73c55537 P f47490f7e2784f42b1b1330aab8ec5e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:39:42.864496 25390 ts_tablet_manager.cc:1905] T 7b017ed58d4847428c59ff3d73c55537 P f47490f7e2784f42b1b1330aab8ec5e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:39:42.878634 25384 ts_tablet_manager.cc:1905] T 7b017ed58d4847428c59ff3d73c55537 P 992b611ae5674367bdcfc1c966a3d914: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:39:42.880602 25384 ts_tablet_manager.cc:1918] T 7b017ed58d4847428c59ff3d73c55537 P 992b611ae5674367bdcfc1c966a3d914: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:39:42.880703 25384 log.cc:1196] T 7b017ed58d4847428c59ff3d73c55537 P 992b611ae5674367bdcfc1c966a3d914: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-2/wal/wals/7b017ed58d4847428c59ff3d73c55537
I20260812 16:39:42.880995 25384 ts_tablet_manager.cc:1939] T 7b017ed58d4847428c59ff3d73c55537 P 992b611ae5674367bdcfc1c966a3d914: Deleting consensus metadata
I20260812 16:39:42.881479 24370 catalog_manager.cc:4841] TS 992b611ae5674367bdcfc1c966a3d914 (127.0.253.195:44745): tablet 7b017ed58d4847428c59ff3d73c55537 (table test-table [id=69cd702c5802434c8ed2a1d16739ee24]) successfully deleted
I20260812 16:39:42.881721 25390 ts_tablet_manager.cc:1918] T 7b017ed58d4847428c59ff3d73c55537 P f47490f7e2784f42b1b1330aab8ec5e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:39:42.881815 25390 log.cc:1196] T 7b017ed58d4847428c59ff3d73c55537 P f47490f7e2784f42b1b1330aab8ec5e7: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552619305671-1015-0/minicluster-data/ts-1/wal/wals/7b017ed58d4847428c59ff3d73c55537
I20260812 16:39:42.882113 25390 ts_tablet_manager.cc:1939] T 7b017ed58d4847428c59ff3d73c55537 P f47490f7e2784f42b1b1330aab8ec5e7: Deleting consensus metadata
I20260812 16:39:42.882598 24368 catalog_manager.cc:4841] TS f47490f7e2784f42b1b1330aab8ec5e7 (127.0.253.194:34169): tablet 7b017ed58d4847428c59ff3d73c55537 (table test-table [id=69cd702c5802434c8ed2a1d16739ee24]) successfully deleted
W20260812 16:39:42.887727 24368 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.253.193:35007 (error 108)
I20260812 16:39:42.888550 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 24771
I20260812 16:39:42.900746 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 25023
I20260812 16:39:42.905016 25333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:42.904849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:35007 (local address 127.0.0.1:53838)
0812 16:39:42.904919 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:42.904968 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.193:35007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 16:39:42.905182 24368 catalog_manager.cc:4584] TS 2a33ab44b895425f8b5cd2a4957728a1 (127.0.253.193:35007): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7b017ed58d4847428c59ff3d73c55537: Network error: Client connection negotiation failed: client connection to 127.0.253.193:35007: connect: Connection refused (error 111)
I20260812 16:39:42.916680 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 24328
I20260812 16:39:42.935487 1015 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 16:39:43.282797 1015 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 16:39:43.282908 1015 mini_ranger.cc:66] Stopping Ranger...
I20260812 16:39:43.637585 1015 mini_ranger.cc:68] Stopped Ranger
2026-08-12 16:39:43.637 UTC [22606] LOG: received smart shutdown request
2026-08-12 16:39:43.642 UTC [22606] LOG: background worker "logical replication launcher" (PID 22633) exited with exit code 1
2026-08-12 16:39:43.642 UTC [22614] LOG: shutting down
2026-08-12 16:39:43.810 UTC [22606] LOG: database system is shut down
2026-08-12T16:39:43Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50738 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:39:43 dist-test-slave-w4v5 krb5kdc[25410](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:39:43 dist-test-slave-w4v5 krb5kdc[25410](info): set up 2 sockets
Aug 12 16:39:43 dist-test-slave-w4v5 krb5kdc[25410](info): commencing operation
krb5kdc: starting...
W20260812 16:39:45.262946 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.305s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:39:45 dist-test-slave-w4v5 krb5kdc[25410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:39:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:39:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
I20260812 16:39:45.396004 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:46709
--webserver_interface=127.0.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:41013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:46709,127.0.253.253:43057,127.0.253.252:33033
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:39:45.480572 25429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:39:45.480783 25429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:39:45.480806 25429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:39:45.482035 25429 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:39:45.482081 25429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:39:45.482096 25429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:39:45.482108 25429 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:39:45.482177 25429 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:39:45.483752 25429 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:41013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:46709,127.0.253.253:43057,127.0.253.252:33033
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:46709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:39:45.484042 25429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:39:45.484309 25429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:39:45.489588 25440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:39:45.489570 25437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:39:45.489570 25444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:39:45.489835 25429 server_base.cc:1034] running on GCE node
I20260812 16:39:45.490103 25429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:39:45.490377 25429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:39:45.491549 25429 hybrid_clock.cc:648] HybridClock initialized: now 1786552785491527 us; error 45 us; skew 500 ppm
Aug 12 16:39:45 dist-test-slave-w4v5 krb5kdc[25410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552785, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:39:45.497876 25429 init.cc:377] Logged in from keytab as kudu/127.0.253.254@KRBTEST.COM (short username kudu)
I20260812 16:39:45.498358 25429 webserver.cc:460] Webserver started at http://127.0.253.254:44977/ using document root <none> and password file <none>
I20260812 16:39:45.498580 25429 fs_manager.cc:359] Metadata directory not provided
I20260812 16:39:45.498624 25429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:39:45.498768 25429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:39:45.499892 25429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "8ca56d74ff164da9b468da1b54080881"
format_stamp: "Formatted at 2026-08-12 16:39:45 on dist-test-slave-w4v5"
server_key: "9867b21b88fc9fdafe25f0556f088507"
server_key_iv: "4dbe26824bfe05fd1bec9510d6a78bbc"
server_key_version: "encryptionkey@0"
I20260812 16:39:45.500201 25429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "8ca56d74ff164da9b468da1b54080881"
format_stamp: "Formatted at 2026-08-12 16:39:45 on dist-test-slave-w4v5"
server_key: "9867b21b88fc9fdafe25f0556f088507"
server_key_iv: "4dbe26824bfe05fd1bec9510d6a78bbc"
server_key_version: "encryptionkey@0"
I20260812 16:39:45.501518 25429 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 16:39:45.502315 25455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:45.502526 25429 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:39:45.502625 25429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "8ca56d74ff164da9b468da1b54080881"
format_stamp: "Formatted at 2026-08-12 16:39:45 on dist-test-slave-w4v5"
server_key: "9867b21b88fc9fdafe25f0556f088507"
server_key_iv: "4dbe26824bfe05fd1bec9510d6a78bbc"
server_key_version: "encryptionkey@0"
I20260812 16:39:45.502691 25429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:39:45.521016 25429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:39:45.523478 25429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:39:45.523597 25429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:39:45.527681 25429 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:46709
I20260812 16:39:45.527725 25547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:46709 every 8 connection(s)
I20260812 16:39:45.528115 25429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:39:45.529139 25548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:39:45.530813 25548 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.253.254" port: 46709 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:39:45.531836 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 25429
I20260812 16:39:45.531965 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:39:45.532146 1015 external_mini_cluster.cc:1428] Setting key b24d9831a2d6b5f0d40fda7f4522af2d
Aug 12 16:39:45 dist-test-slave-w4v5 krb5kdc[25410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552785, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.253.253@KRBTEST.COM; defaulting to no policy
I20260812 16:39:45.541409 25553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:45.531587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49696 (local address 127.0.253.254:46709)
0812 16:39:45.531801 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:45.531806 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:45.531839 (+ 33us) server_negotiation.cc:407] Connection header received
0812 16:39:45.532526 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:45.532528 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:45.532554 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:45.532654 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:39:45.533239 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.534066 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.535231 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.535556 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.535698 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:45.537945 (+ 2247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:39:45.537953 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:39:45.537953 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:39:45.537961 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:39:45.539669 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:45.540081 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:45.540084 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:45.540085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:45.540170 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:45.540409 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:45.540410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:45.540410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:45.540681 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:39:45.540773 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:45.540970 (+ 197us) server_negotiation.cc:299] Negotiation successful
0812 16:39:45.541163 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":186,"thread_start_us":103,"threads_started":1}
I20260812 16:39:45.541419 25552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:45.531535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:46709 (local address 127.0.0.1:49696)
0812 16:39:45.531703 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:45.531752 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 16:39:45.532362 (+ 610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:39:45.532661 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:39:45.532663 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:39:45.532963 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:39:45.533124 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:45.533128 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.534286 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.534288 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:45.535125 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:45.535129 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.535656 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.535657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:45.535724 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:45.536392 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:39:45.536401 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:39:45.537851 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:39:45.539770 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:45.539777 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:45.539779 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:45.539977 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:45.540247 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:45.540247 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:45.540248 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:45.540336 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:45.540763 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:39:45.540769 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:39:45.540835 (+ 66us) client_negotiation.cc:769] Sending connection context
0812 16:39:45.540943 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 16:39:45.541213 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":129,"thread_start_us":70,"threads_started":1}
I20260812 16:39:45.542191 25548 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43057 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:39:45.542579 25552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:45.542462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.253:43057 (local address 127.0.0.1:40530)
0812 16:39:45.542499 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:45.542559 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.253:43057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260812 16:39:45.542726 25461 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.253.253:43057: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.253@KRBTEST.COM" created.
I20260812 16:39:45.543249 25552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:45.543156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.253:43057 (local address 127.0.0.1:40546)
0812 16:39:45.543208 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:45.543234 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.253:43057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260812 16:39:45.543365 25548 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.253.253:43057: Network error: Client connection negotiation failed: client connection to 127.0.253.253:43057: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.253.keytab.
Entry for principal kudu/127.0.253.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.253.keytab.
I20260812 16:39:45.558092 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.253:43057
--webserver_interface=127.0.253.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.253.keytab
--principal=kudu/127.0.253.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:41013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:46709,127.0.253.253:43057,127.0.253.252:33033
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:39:45.592603 25548 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43057 } attempt: 1
I20260812 16:39:45.593209 25552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:45.593048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.253:43057 (local address 127.0.0.1:40548)
0812 16:39:45.593131 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:45.593177 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.253:43057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260812 16:39:45.593667 25552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:45.593517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.253:43057 (local address 127.0.0.1:40562)
0812 16:39:45.593604 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:45.593643 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.253:43057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260812 16:39:45.593799 25548 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.253.253:43057: Network error: Client connection negotiation failed: client connection to 127.0.253.253:43057: connect: Connection refused (error 111)
W20260812 16:39:45.643292 25558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:39:45.643518 25558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:39:45.643564 25558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:39:45.644712 25558 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:39:45.644781 25558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:39:45.644806 25558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:39:45.644836 25558 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:39:45.644867 25558 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:39:45.646369 25558 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:41013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:46709,127.0.253.253:43057,127.0.253.252:33033
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.253.keytab
--principal=kudu/127.0.253.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.253:43057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.253.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25558
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:39:45.646654 25558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:39:45.646888 25558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:39:45.652179 25575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:39:45.652364 25558 server_base.cc:1034] running on GCE node
W20260812 16:39:45.652315 25569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:39:45.652179 25572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:39:45.652772 25558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:39:45.653085 25558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:39:45.654258 25558 hybrid_clock.cc:648] HybridClock initialized: now 1786552785654240 us; error 48 us; skew 500 ppm
Aug 12 16:39:45 dist-test-slave-w4v5 krb5kdc[25410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552785, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:39:45.657021 25558 init.cc:377] Logged in from keytab as kudu/127.0.253.253@KRBTEST.COM (short username kudu)
I20260812 16:39:45.657529 25558 webserver.cc:460] Webserver started at http://127.0.253.253:36593/ using document root <none> and password file <none>
I20260812 16:39:45.657797 25558 fs_manager.cc:359] Metadata directory not provided
I20260812 16:39:45.657871 25558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:39:45.658020 25558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:39:45.659194 25558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-1/data/instance:
uuid: "9130c123b1054d0cac3c54319804999c"
format_stamp: "Formatted at 2026-08-12 16:39:45 on dist-test-slave-w4v5"
server_key: "1f9ab69bfa3174819e75deeac98b14bc"
server_key_iv: "88e03ba1eed24679056468d856b8f2d3"
server_key_version: "encryptionkey@0"
I20260812 16:39:45.659574 25558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-1/wal/instance:
uuid: "9130c123b1054d0cac3c54319804999c"
format_stamp: "Formatted at 2026-08-12 16:39:45 on dist-test-slave-w4v5"
server_key: "1f9ab69bfa3174819e75deeac98b14bc"
server_key_iv: "88e03ba1eed24679056468d856b8f2d3"
server_key_version: "encryptionkey@0"
I20260812 16:39:45.660943 25558 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:39:45.661681 25583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:45.661866 25558 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:39:45.661974 25558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-1/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-1/wal
uuid: "9130c123b1054d0cac3c54319804999c"
format_stamp: "Formatted at 2026-08-12 16:39:45 on dist-test-slave-w4v5"
server_key: "1f9ab69bfa3174819e75deeac98b14bc"
server_key_iv: "88e03ba1eed24679056468d856b8f2d3"
server_key_version: "encryptionkey@0"
I20260812 16:39:45.661964 25548 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43057 } attempt: 2
I20260812 16:39:45.662050 25558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:39:45.662367 25552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:45.662259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.253:43057 (local address 127.0.0.1:40574)
0812 16:39:45.662302 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:45.662340 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.253:43057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260812 16:39:45.662717 25552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:45.662584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.253:43057 (local address 127.0.0.1:40590)
0812 16:39:45.662666 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:45.662697 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.253:43057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260812 16:39:45.662807 25548 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.253.253:43057: Network error: Client connection negotiation failed: client connection to 127.0.253.253:43057: connect: Connection refused (error 111)
I20260812 16:39:45.676246 25558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:39:45.678678 25558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:39:45.678853 25558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:39:45.683235 25558 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.253:43057
I20260812 16:39:45.683295 25695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.253:43057 every 8 connection(s)
I20260812 16:39:45.683668 25558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-1/data/info.pb
I20260812 16:39:45.684576 25700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:39:45.686092 25700 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.253.254" port: 46709 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 16:39:45 dist-test-slave-w4v5 krb5kdc[25410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552785, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.253@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:39:45.693914 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 25558
I20260812 16:39:45.694049 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-1/wal/instance
I20260812 16:39:45.694275 1015 external_mini_cluster.cc:1428] Setting key 35b09cb1d01b5eabb45ff4c0e3a13e96
I20260812 16:39:45.694751 25553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:45.686792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49704 (local address 127.0.253.254:46709)
0812 16:39:45.686839 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:45.686842 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:45.687034 (+ 192us) server_negotiation.cc:407] Connection header received
0812 16:39:45.687532 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:45.687533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:45.687545 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:45.687607 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:39:45.688174 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.688658 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.689588 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.689790 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.689847 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:45.692045 (+ 2198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:39:45.692047 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:39:45.692047 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:39:45.692055 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:39:45.693300 (+ 1245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:45.693661 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:45.693662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:45.693663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:45.693722 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:45.693975 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:45.693976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:45.693976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:45.694135 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:39:45.694232 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:45.694523 (+ 291us) server_negotiation.cc:299] Negotiation successful
0812 16:39:45.694623 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 16:39:45.694770 25704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:45.686740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:46709 (local address 127.0.0.1:49704)
0812 16:39:45.686904 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:45.686947 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 16:39:45.687447 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:39:45.687665 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:39:45.687668 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:39:45.687908 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:39:45.688098 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:45.688101 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.688741 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.688741 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:45.689508 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:45.689510 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.689865 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.689866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:45.689965 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:45.690470 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:39:45.690478 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:39:45.691955 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:39:45.693372 (+ 1417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:45.693378 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:45.693380 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:45.693596 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:45.693808 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:45.693809 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:45.693809 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:45.693907 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:45.694222 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:39:45.694228 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:39:45.694350 (+ 122us) client_negotiation.cc:769] Sending connection context
0812 16:39:45.694477 (+ 127us) client_negotiation.cc:240] Negotiation successful
0812 16:39:45.694600 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":88,"threads_started":1}
I20260812 16:39:45.695549 25700 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43057 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 16:39:45 dist-test-slave-w4v5 krb5kdc[25410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552785, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.253@KRBTEST.COM for kudu/127.0.253.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.253.252@KRBTEST.COM; defaulting to no policy
I20260812 16:39:45.702809 25704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:45.695843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.253:43057 (local address 127.0.0.1:40598)
0812 16:39:45.695891 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:45.695901 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:39:45.695939 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:39:45.696201 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:39:45.696202 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:39:45.696342 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:39:45.696385 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:45.696386 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.697264 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.697265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:45.697766 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:45.697768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.698264 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.698265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:45.698336 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:45.698775 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:39:45.698780 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:39:45.699996 (+ 1216us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:39:45.701748 (+ 1752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:45.701749 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:45.701750 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:45.701919 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:45.702139 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:45.702140 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:45.702141 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:45.702196 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:45.702491 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:39:45.702492 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:39:45.702529 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 16:39:45.702592 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 16:39:45.702703 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 16:39:45.702924 25705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:45.695909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40598 (local address 127.0.253.253:43057)
0812 16:39:45.696071 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:45.696075 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:45.696090 (+ 15us) server_negotiation.cc:407] Connection header received
0812 16:39:45.696107 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:45.696108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:45.696118 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:45.696202 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:39:45.696467 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.697138 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.697862 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.698173 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.698268 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:45.700083 (+ 1815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:39:45.700092 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:39:45.700092 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:39:45.700099 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:39:45.701643 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:45.702012 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:45.702015 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:45.702015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:45.702071 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:45.702268 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:45.702269 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:45.702270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:45.702412 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:39:45.702503 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:45.702619 (+ 116us) server_negotiation.cc:299] Negotiation successful
0812 16:39:45.702794 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":87,"server-negotiator.queue_time_us":136,"thread_start_us":84,"threads_started":1}
I20260812 16:39:45.703590 25700 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.253.252" port: 33033 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:39:45.703946 25704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:45.703841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:33033 (local address 127.0.0.1:43870)
0812 16:39:45.703886 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:45.703929 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.252:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 16:39:45.704087 25591 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.253.252:33033: connect: Connection refused (error 111)
I20260812 16:39:45.704463 25704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:45.704405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:33033 (local address 127.0.0.1:43872)
0812 16:39:45.704427 (+ 22us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:45.704449 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.252:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11}
W20260812 16:39:45.704571 25700 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.253.252:33033: Network error: Client connection negotiation failed: client connection to 127.0.253.252:33033: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.252.keytab.
Entry for principal kudu/127.0.253.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.252.keytab.
I20260812 16:39:45.722966 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.252:33033
--webserver_interface=127.0.253.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.252.keytab
--principal=kudu/127.0.253.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:41013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:46709,127.0.253.253:43057,127.0.253.252:33033
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:39:45.753829 25700 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.253.252" port: 33033 } attempt: 1
I20260812 16:39:45.753962 25548 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43057 } attempt: 3
I20260812 16:39:45.754395 25704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:45.754234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:33033 (local address 127.0.0.1:43876)
0812 16:39:45.754308 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:45.754352 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.252:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260812 16:39:45.754813 25704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:45.754690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:33033 (local address 127.0.0.1:43892)
0812 16:39:45.754748 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:45.754783 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.252:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 16:39:45.754972 25700 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.253.252:33033: Network error: Client connection negotiation failed: client connection to 127.0.253.252:33033: connect: Connection refused (error 111)
Aug 12 16:39:45 dist-test-slave-w4v5 krb5kdc[25410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552785, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for kudu/127.0.253.253@KRBTEST.COM
I20260812 16:39:45.761857 25705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:45.754359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40608 (local address 127.0.253.253:43057)
0812 16:39:45.754390 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:45.754392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:45.754404 (+ 12us) server_negotiation.cc:407] Connection header received
0812 16:39:45.754485 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:45.754486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:45.754505 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:45.755039 (+ 534us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:39:45.755078 (+ 39us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.755632 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.756652 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.756965 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.757038 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:45.759197 (+ 2159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:39:45.759198 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:39:45.759199 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:39:45.759206 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:39:45.760662 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:45.761039 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:45.761041 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:45.761041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:45.761114 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:45.761326 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:45.761327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:45.761327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:45.761445 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:39:45.761521 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:45.761698 (+ 177us) server_negotiation.cc:299] Negotiation successful
0812 16:39:45.761772 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 16:39:45.761864 25552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:45.754293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.253:43057 (local address 127.0.0.1:40608)
0812 16:39:45.754352 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:45.754362 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:39:45.754405 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:39:45.754634 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:39:45.754636 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:39:45.754831 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:39:45.754957 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:45.754960 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.755722 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.755723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:45.756537 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:45.756541 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.757037 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.757038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:45.757137 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:45.757726 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:39:45.757732 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:39:45.759039 (+ 1307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:39:45.760771 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:45.760773 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:45.760773 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:45.760951 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:45.761194 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:45.761195 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:45.761195 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:45.761260 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:45.761523 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:39:45.761524 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:39:45.761581 (+ 57us) client_negotiation.cc:769] Sending connection context
0812 16:39:45.761659 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 16:39:45.761766 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 16:39:45.762275 25548 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.253.252" port: 33033 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:39:45.762684 25552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:45.762552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:33033 (local address 127.0.0.1:43908)
0812 16:39:45.762626 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:45.762660 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.252:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260812 16:39:45.763043 25552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:45.762982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:33033 (local address 127.0.0.1:43912)
0812 16:39:45.763005 (+ 23us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:45.763027 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.252:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260812 16:39:45.763163 25548 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.253.252:33033: Network error: Client connection negotiation failed: client connection to 127.0.253.252:33033: connect: Connection refused (error 111)
I20260812 16:39:45.794334 25548 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.253.252" port: 33033 } attempt: 1
I20260812 16:39:45.794911 25552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:45.794762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:33033 (local address 127.0.0.1:43922)
0812 16:39:45.794836 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:45.794878 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.252:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 16:39:45.795310 25552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:45.795206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:33033 (local address 127.0.0.1:43938)
0812 16:39:45.795258 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:45.795292 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.252:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 16:39:45.795410 25548 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.253.252:33033: Network error: Client connection negotiation failed: client connection to 127.0.253.252:33033: connect: Connection refused (error 111)
W20260812 16:39:45.810791 25712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:39:45.811010 25712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:39:45.811035 25712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:39:45.812269 25712 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:39:45.812317 25712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:39:45.812332 25712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:39:45.812346 25712 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:39:45.812358 25712 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:39:45.813972 25712 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:41013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:46709,127.0.253.253:43057,127.0.253.252:33033
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.252.keytab
--principal=kudu/127.0.253.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.252:33033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.253.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:39:45.814258 25712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:39:45.814517 25712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:39:45.820595 25723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:39:45.820744 25712 server_base.cc:1034] running on GCE node
W20260812 16:39:45.820650 25727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:39:45.820590 25722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:39:45.821164 25712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:39:45.821535 25712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:39:45.822749 25712 hybrid_clock.cc:648] HybridClock initialized: now 1786552785822728 us; error 45 us; skew 500 ppm
I20260812 16:39:45.823148 25700 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.253.252" port: 33033 } attempt: 2
I20260812 16:39:45.823598 25704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:45.823436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:33033 (local address 127.0.0.1:43946)
0812 16:39:45.823512 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:45.823558 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.252:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 16:39:45.824064 25704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:45.823912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:33033 (local address 127.0.0.1:43952)
0812 16:39:45.823974 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:45.824031 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.252:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260812 16:39:45.824201 25700 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.253.252:33033: Network error: Client connection negotiation failed: client connection to 127.0.253.252:33033: connect: Connection refused (error 111)
Aug 12 16:39:45 dist-test-slave-w4v5 krb5kdc[25410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552785, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:39:45.825927 25712 init.cc:377] Logged in from keytab as kudu/127.0.253.252@KRBTEST.COM (short username kudu)
I20260812 16:39:45.826493 25712 webserver.cc:460] Webserver started at http://127.0.253.252:39683/ using document root <none> and password file <none>
I20260812 16:39:45.826784 25712 fs_manager.cc:359] Metadata directory not provided
I20260812 16:39:45.826859 25712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:39:45.827032 25712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:39:45.828320 25712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-2/data/instance:
uuid: "e7b688e171914559b284299db6c81f3d"
format_stamp: "Formatted at 2026-08-12 16:39:45 on dist-test-slave-w4v5"
server_key: "529c78f80e3aee21bfccf5589d296d8b"
server_key_iv: "d37bc97ee55e0a776a17d22926396b5d"
server_key_version: "encryptionkey@0"
I20260812 16:39:45.828735 25712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-2/wal/instance:
uuid: "e7b688e171914559b284299db6c81f3d"
format_stamp: "Formatted at 2026-08-12 16:39:45 on dist-test-slave-w4v5"
server_key: "529c78f80e3aee21bfccf5589d296d8b"
server_key_iv: "d37bc97ee55e0a776a17d22926396b5d"
server_key_version: "encryptionkey@0"
I20260812 16:39:45.830336 25712 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 16:39:45.831354 25743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:45.831600 25712 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:39:45.831712 25712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-2/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-2/wal
uuid: "e7b688e171914559b284299db6c81f3d"
format_stamp: "Formatted at 2026-08-12 16:39:45 on dist-test-slave-w4v5"
server_key: "529c78f80e3aee21bfccf5589d296d8b"
server_key_iv: "d37bc97ee55e0a776a17d22926396b5d"
server_key_version: "encryptionkey@0"
I20260812 16:39:45.831800 25712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:39:45.840584 25712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:39:45.842839 25712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:39:45.843020 25712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:39:45.847235 25712 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.252:33033
I20260812 16:39:45.847294 25840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.252:33033 every 8 connection(s)
I20260812 16:39:45.847689 25712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-2/data/info.pb
I20260812 16:39:45.848599 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 25712
I20260812 16:39:45.848623 25841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:39:45.848752 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552619305671-1015-0/minicluster-data/master-2/wal/instance
I20260812 16:39:45.848968 1015 external_mini_cluster.cc:1428] Setting key 78b652d22410c40b95e6df72b70347a1
I20260812 16:39:45.851217 25841 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.253.254" port: 46709 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 16:39:45 dist-test-slave-w4v5 krb5kdc[25410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.253.252@KRBTEST.COM
Aug 12 16:39:45 dist-test-slave-w4v5 krb5kdc[25410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552785, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.252@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:39:45.859967 25850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:45.849568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43964 (local address 127.0.253.252:33033)
0812 16:39:45.849814 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:45.849824 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:45.849859 (+ 35us) server_negotiation.cc:407] Connection header received
0812 16:39:45.850385 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:45.850395 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:45.850463 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:45.850644 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:39:45.851331 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.852460 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.853227 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.853658 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.853784 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:45.856386 (+ 2602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:39:45.856393 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:39:45.856394 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:39:45.856400 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:39:45.858108 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:45.858518 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:45.858520 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:45.858521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:45.858610 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:45.858879 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:45.858880 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:45.858880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:45.859260 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:39:45.859391 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:45.859525 (+ 134us) server_negotiation.cc:299] Negotiation successful
0812 16:39:45.859729 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":96,"threads_started":1}
I20260812 16:39:45.860189 25852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:45.851917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:46709 (local address 127.0.0.1:49708)
0812 16:39:45.852157 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:45.852199 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 16:39:45.852287 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:39:45.852467 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:39:45.852472 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:39:45.852758 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:39:45.852826 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:45.852828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.853489 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.853490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:45.854096 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:45.854098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.854500 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.854501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:45.854581 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:45.855133 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:39:45.855144 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:39:45.856910 (+ 1766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:39:45.858887 (+ 1977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:45.858894 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:45.858897 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:45.859218 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:45.859469 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:45.859470 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:45.859470 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:45.859533 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:45.859831 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:39:45.859836 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:39:45.859885 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 16:39:45.859978 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 16:39:45.860099 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":124,"threads_started":1}
I20260812 16:39:45.860201 25553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:45.852000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49708 (local address 127.0.253.254:46709)
0812 16:39:45.852082 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:45.852084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:45.852267 (+ 183us) server_negotiation.cc:407] Connection header received
0812 16:39:45.852358 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:45.852359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:45.852370 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:45.852445 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:39:45.852929 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.853408 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.854192 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.854397 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.854485 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:45.857007 (+ 2522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:39:45.857009 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:39:45.857009 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:39:45.857017 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:39:45.858805 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:45.859324 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:45.859326 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:45.859326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:45.859394 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:45.859638 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:45.859640 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:45.859640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:45.859778 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:39:45.859833 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:45.859995 (+ 162us) server_negotiation.cc:299] Negotiation successful
0812 16:39:45.860103 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 16:39:45.860862 25841 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43057 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 16:39:45 dist-test-slave-w4v5 krb5kdc[25410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552785, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.252@KRBTEST.COM for kudu/127.0.253.253@KRBTEST.COM
Aug 12 16:39:45 dist-test-slave-w4v5 krb5kdc[25410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:39:45.867789 25852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:45.861108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.253:43057 (local address 127.0.0.1:40618)
0812 16:39:45.861160 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:45.861171 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:39:45.861208 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:39:45.861356 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:39:45.861356 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:39:45.861511 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:39:45.861554 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:45.861555 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.862161 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.862162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:45.862708 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:45.862710 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.863196 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.863197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:45.863255 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:45.863666 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:39:45.863670 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:39:45.864928 (+ 1258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:39:45.866702 (+ 1774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:45.866703 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:45.866704 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:45.866879 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:45.867130 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:45.867131 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:45.867131 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:45.867192 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:45.867465 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:39:45.867466 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:39:45.867509 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 16:39:45.867595 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 16:39:45.867693 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 16:39:45.867790 25705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:45.861200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40618 (local address 127.0.253.253:43057)
0812 16:39:45.861241 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:45.861243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:45.861250 (+ 7us) server_negotiation.cc:407] Connection header received
0812 16:39:45.861274 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:45.861275 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:45.861286 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:45.861334 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:39:45.861626 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.862064 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.862797 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.863135 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.863204 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:45.865023 (+ 1819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:39:45.865024 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:39:45.865024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:39:45.865031 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:39:45.866618 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:45.866956 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:45.866958 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:45.866958 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:45.867024 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:45.867269 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:45.867270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:45.867270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:45.867389 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:39:45.867462 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:45.867630 (+ 168us) server_negotiation.cc:299] Negotiation successful
0812 16:39:45.867702 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 16:39:45.868388 25553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:45.860829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49724 (local address 127.0.253.254:46709)
0812 16:39:45.860860 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:45.860862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:45.860870 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:39:45.860919 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:45.860920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:45.860929 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:45.860985 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:39:45.861781 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.862271 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.862996 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.863259 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.863334 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:45.865782 (+ 2448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:39:45.865784 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:39:45.865784 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:39:45.865791 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:39:45.867241 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:45.867594 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:45.867595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:45.867596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:45.867656 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:45.867844 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:45.867845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:45.867845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:45.867960 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:39:45.868176 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:45.868218 (+ 42us) server_negotiation.cc:299] Negotiation successful
0812 16:39:45.868292 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260812 16:39:45.868742 25841 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.253.252" port: 33033 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:39:45.870594 25548 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.253.252" port: 33033 } attempt: 2
Aug 12 16:39:45 dist-test-slave-w4v5 krb5kdc[25410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552785, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.252@KRBTEST.COM for kudu/127.0.253.252@KRBTEST.COM
Aug 12 16:39:45 dist-test-slave-w4v5 krb5kdc[25410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.253.253@KRBTEST.COM
Aug 12 16:39:45 dist-test-slave-w4v5 krb5kdc[25410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552785, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for kudu/127.0.253.252@KRBTEST.COM
I20260812 16:39:45.876329 25852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:45.869006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:33033 (local address 127.0.0.1:43970)
0812 16:39:45.869055 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:45.869065 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:39:45.869101 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:39:45.869248 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:39:45.869249 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:39:45.869427 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:39:45.869464 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:45.869466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.870392 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.870393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:45.871196 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:45.871199 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.871679 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.871680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:45.871736 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:45.872151 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:39:45.872156 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:39:45.873551 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:39:45.875287 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:45.875288 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:45.875288 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:45.875453 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:45.875658 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:45.875659 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:45.875659 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:45.875722 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:45.875990 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:39:45.875990 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:39:45.876029 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 16:39:45.876118 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 16:39:45.876220 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 16:39:45.876766 25850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:45.869075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43970 (local address 127.0.253.252:33033)
0812 16:39:45.869115 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:45.869116 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:45.869124 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:39:45.869146 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:45.869146 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:45.869155 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:45.869243 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:39:45.869529 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.870299 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.871283 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.871578 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.871636 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:45.873620 (+ 1984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:39:45.873621 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:39:45.873621 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:39:45.873627 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:39:45.875188 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:45.875538 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:45.875539 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:45.875540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:45.875597 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:45.875789 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:45.875789 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:45.875790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:45.875911 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:39:45.875969 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:45.876151 (+ 182us) server_negotiation.cc:299] Negotiation successful
0812 16:39:45.876703 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":21}
I20260812 16:39:45.878535 25841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7b688e171914559b284299db6c81f3d: Bootstrap starting.
I20260812 16:39:45.878680 25705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:45.869493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40632 (local address 127.0.253.253:43057)
0812 16:39:45.869559 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:45.869561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:45.869589 (+ 28us) server_negotiation.cc:407] Connection header received
0812 16:39:45.869665 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:45.869666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:45.869673 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:45.869712 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:39:45.870268 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.871233 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.872461 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.872744 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.872788 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:45.875484 (+ 2696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:39:45.875485 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:39:45.875485 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:39:45.875492 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:39:45.877310 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:45.877699 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:45.877700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:45.877700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:45.877759 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:45.878012 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:45.878013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:45.878013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:45.878145 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:39:45.878378 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:45.878420 (+ 42us) server_negotiation.cc:299] Negotiation successful
0812 16:39:45.878593 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 16:39:45.879165 25552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:45.870871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:33033 (local address 127.0.0.1:43972)
0812 16:39:45.870929 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:45.870939 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:39:45.870982 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:39:45.871413 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:39:45.871414 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:39:45.871553 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:39:45.871584 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:45.871586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.872554 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.872556 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:45.873032 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:45.873055 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.873556 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.873557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:45.873633 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:45.874004 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:39:45.874010 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:39:45.875900 (+ 1890us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:39:45.877954 (+ 2054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:45.877955 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:45.877956 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:45.878166 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:45.878386 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:45.878387 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:45.878387 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:45.878446 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:45.878807 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:39:45.878808 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:39:45.878857 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 16:39:45.878907 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 16:39:45.879010 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 16:39:45.879462 25841 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7b688e171914559b284299db6c81f3d: Neither blocks nor log segments found. Creating new log.
I20260812 16:39:45.879611 25854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:45.871057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43972 (local address 127.0.253.252:33033)
0812 16:39:45.871276 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:45.871278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:45.871285 (+ 7us) server_negotiation.cc:407] Connection header received
0812 16:39:45.871303 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:45.871304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:45.871319 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:45.871392 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:39:45.871683 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.872463 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.873146 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.873464 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.873538 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:45.875994 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:39:45.875995 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:39:45.875996 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:39:45.876003 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:39:45.877881 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:45.878248 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:45.878249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:45.878249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:45.878308 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:45.878526 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:45.878527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:45.878528 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:45.878714 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:39:45.879284 (+ 570us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:45.879341 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 16:39:45.879426 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":173,"threads_started":1}
I20260812 16:39:45.879864 25841 log.cc:824] T 00000000000000000000000000000000 P e7b688e171914559b284299db6c81f3d: Log is configured to *not* fsync() on all Append() calls
I20260812 16:39:45.880182 1015 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:39:45.881239 25548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ca56d74ff164da9b468da1b54080881: Bootstrap starting.
I20260812 16:39:45.882017 25548 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ca56d74ff164da9b468da1b54080881: Neither blocks nor log segments found. Creating new log.
I20260812 16:39:45.882376 25548 log.cc:824] T 00000000000000000000000000000000 P 8ca56d74ff164da9b468da1b54080881: Log is configured to *not* fsync() on all Append() calls
I20260812 16:39:45.883029 25841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7b688e171914559b284299db6c81f3d: No bootstrap required, opened a new log
I20260812 16:39:45.884310 25548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ca56d74ff164da9b468da1b54080881: No bootstrap required, opened a new log
I20260812 16:39:45.885151 25841 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7b688e171914559b284299db6c81f3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ca56d74ff164da9b468da1b54080881" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 46709 } } peers { permanent_uuid: "9130c123b1054d0cac3c54319804999c" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43057 } } peers { permanent_uuid: "e7b688e171914559b284299db6c81f3d" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 33033 } }
I20260812 16:39:45.885310 25841 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7b688e171914559b284299db6c81f3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:39:45.885351 25841 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7b688e171914559b284299db6c81f3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7b688e171914559b284299db6c81f3d, State: Initialized, Role: FOLLOWER
I20260812 16:39:45.885504 25841 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7b688e171914559b284299db6c81f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ca56d74ff164da9b468da1b54080881" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 46709 } } peers { permanent_uuid: "9130c123b1054d0cac3c54319804999c" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43057 } } peers { permanent_uuid: "e7b688e171914559b284299db6c81f3d" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 33033 } }
I20260812 16:39:45.885861 25841 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7b688e171914559b284299db6c81f3d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:39:45.886435 25548 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ca56d74ff164da9b468da1b54080881 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ca56d74ff164da9b468da1b54080881" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 46709 } } peers { permanent_uuid: "9130c123b1054d0cac3c54319804999c" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43057 } } peers { permanent_uuid: "e7b688e171914559b284299db6c81f3d" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 33033 } }
I20260812 16:39:45.886564 25548 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ca56d74ff164da9b468da1b54080881 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:39:45.886602 25548 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ca56d74ff164da9b468da1b54080881 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ca56d74ff164da9b468da1b54080881, State: Initialized, Role: FOLLOWER
W20260812 16:39:45.888427 25893 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e7b688e171914559b284299db6c81f3d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 16:39:45.888491 25893 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:39:45.888857 25871 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7b688e171914559b284299db6c81f3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ca56d74ff164da9b468da1b54080881" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 46709 } } peers { permanent_uuid: "9130c123b1054d0cac3c54319804999c" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43057 } } peers { permanent_uuid: "e7b688e171914559b284299db6c81f3d" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 33033 } } }
I20260812 16:39:45.889056 25871 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7b688e171914559b284299db6c81f3d [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:39:45.887679 25548 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ca56d74ff164da9b468da1b54080881 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ca56d74ff164da9b468da1b54080881" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 46709 } } peers { permanent_uuid: "9130c123b1054d0cac3c54319804999c" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43057 } } peers { permanent_uuid: "e7b688e171914559b284299db6c81f3d" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 33033 } }
I20260812 16:39:45.889966 25548 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ca56d74ff164da9b468da1b54080881 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:39:45.888135 25712 master_runner.cc:186] Error getting master registration for 127.0.253.254:46709: OK, instance_id { permanent_uuid: "8ca56d74ff164da9b468da1b54080881" instance_seqno: 1786552785526417 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 16:39:45.890709 25712 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 16:39:45.891680 25705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:45.881723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40648 (local address 127.0.253.253:43057)
0812 16:39:45.881761 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:45.881763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:45.882300 (+ 537us) server_negotiation.cc:407] Connection header received
0812 16:39:45.882403 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:45.882404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:45.882414 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:45.882445 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:39:45.883227 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.883876 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.884866 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.885155 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.885203 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:45.887291 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:39:45.887293 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:39:45.887293 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:39:45.887300 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:39:45.889669 (+ 2369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:45.890077 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:45.890079 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:45.890079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:45.890156 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:45.890362 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:45.890363 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:45.890364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:45.890517 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:39:45.891202 (+ 685us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:45.891489 (+ 287us) server_negotiation.cc:299] Negotiation successful
0812 16:39:45.891574 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 16:39:45.892844 25553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:45.881575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49740 (local address 127.0.253.254:46709)
0812 16:39:45.881621 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:45.881623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:45.881726 (+ 103us) server_negotiation.cc:407] Connection header received
0812 16:39:45.881827 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:45.881828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:45.881839 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:45.881868 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:39:45.882708 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.883480 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.885327 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.885606 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.885647 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:45.887245 (+ 1598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:39:45.887246 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:39:45.887247 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:39:45.887254 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:39:45.891482 (+ 4228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:45.891869 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:45.891870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:45.891870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:45.891933 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:45.892165 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:45.892166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:45.892166 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:45.892304 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:39:45.892373 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:45.892647 (+ 274us) server_negotiation.cc:299] Negotiation successful
0812 16:39:45.892733 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 16:39:45.893740 25854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:45.881653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43980 (local address 127.0.253.252:33033)
0812 16:39:45.881687 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:45.881700 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:45.881981 (+ 281us) server_negotiation.cc:407] Connection header received
0812 16:39:45.882028 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:45.882029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:45.882038 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:45.882091 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:39:45.882839 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.883553 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.886758 (+ 3205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.887113 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.887185 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:45.889078 (+ 1893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:39:45.889079 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:39:45.889080 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:39:45.889086 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:39:45.890767 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:45.891182 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:45.891183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:45.891184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:45.891239 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:45.892828 (+ 1589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:45.892830 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:45.892830 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:45.892981 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:39:45.893073 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:45.893565 (+ 492us) server_negotiation.cc:299] Negotiation successful
0812 16:39:45.893646 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260812 16:39:45.894506 25917 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8ca56d74ff164da9b468da1b54080881: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 16:39:45.894644 25917 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:39:45.894833 25895 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ca56d74ff164da9b468da1b54080881 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ca56d74ff164da9b468da1b54080881" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 46709 } } peers { permanent_uuid: "9130c123b1054d0cac3c54319804999c" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43057 } } peers { permanent_uuid: "e7b688e171914559b284299db6c81f3d" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 33033 } } }
I20260812 16:39:45.894955 25895 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ca56d74ff164da9b468da1b54080881 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:39:45.894313 25429 master_runner.cc:186] Error getting master registration for 127.0.253.253:43057: OK, instance_id { permanent_uuid: "9130c123b1054d0cac3c54319804999c" instance_seqno: 1786552785681933 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 16:39:45.896711 25429 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 16:39:45.898038 25871 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7b688e171914559b284299db6c81f3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:39:45.898291 25871 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7b688e171914559b284299db6c81f3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ca56d74ff164da9b468da1b54080881" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 46709 } } peers { permanent_uuid: "9130c123b1054d0cac3c54319804999c" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43057 } } peers { permanent_uuid: "e7b688e171914559b284299db6c81f3d" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 33033 } }
I20260812 16:39:45.898917 25507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e7b688e171914559b284299db6c81f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ca56d74ff164da9b468da1b54080881" is_pre_election: true
I20260812 16:39:45.899049 25507 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8ca56d74ff164da9b468da1b54080881 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7b688e171914559b284299db6c81f3d in term 0.
I20260812 16:39:45.899307 25748 leader_election.cc:302] T 00000000000000000000000000000000 P e7b688e171914559b284299db6c81f3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ca56d74ff164da9b468da1b54080881, e7b688e171914559b284299db6c81f3d; no voters:
I20260812 16:39:45.899344 25871 leader_election.cc:288] T 00000000000000000000000000000000 P e7b688e171914559b284299db6c81f3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ca56d74ff164da9b468da1b54080881 (127.0.253.254:46709), 9130c123b1054d0cac3c54319804999c (127.0.253.253:43057)
I20260812 16:39:45.899458 25871 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7b688e171914559b284299db6c81f3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:39:45.899531 25871 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7b688e171914559b284299db6c81f3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:39:45.899652 25871 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7b688e171914559b284299db6c81f3d [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:39:45.901150 25871 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7b688e171914559b284299db6c81f3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ca56d74ff164da9b468da1b54080881" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 46709 } } peers { permanent_uuid: "9130c123b1054d0cac3c54319804999c" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43057 } } peers { permanent_uuid: "e7b688e171914559b284299db6c81f3d" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 33033 } }
I20260812 16:39:45.901530 25507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e7b688e171914559b284299db6c81f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ca56d74ff164da9b468da1b54080881"
I20260812 16:39:45.901625 25507 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ca56d74ff164da9b468da1b54080881 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:39:45.901598 25871 leader_election.cc:288] T 00000000000000000000000000000000 P e7b688e171914559b284299db6c81f3d [CANDIDATE]: Term 1 election: Requested vote from peers 8ca56d74ff164da9b468da1b54080881 (127.0.253.254:46709), 9130c123b1054d0cac3c54319804999c (127.0.253.253:43057)
I20260812 16:39:45.902524 25507 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8ca56d74ff164da9b468da1b54080881 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7b688e171914559b284299db6c81f3d in term 1.
I20260812 16:39:45.902737 25748 leader_election.cc:302] T 00000000000000000000000000000000 P e7b688e171914559b284299db6c81f3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ca56d74ff164da9b468da1b54080881, e7b688e171914559b284299db6c81f3d; no voters:
I20260812 16:39:45.902881 25871 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7b688e171914559b284299db6c81f3d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:39:45.903016 25871 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7b688e171914559b284299db6c81f3d [term 1 LEADER]: Becoming Leader. State: Replica: e7b688e171914559b284299db6c81f3d, State: Running, Role: LEADER
I20260812 16:39:45.903250 25871 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7b688e171914559b284299db6c81f3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ca56d74ff164da9b468da1b54080881" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 46709 } } peers { permanent_uuid: "9130c123b1054d0cac3c54319804999c" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43057 } } peers { permanent_uuid: "e7b688e171914559b284299db6c81f3d" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 33033 } }
I20260812 16:39:45.905063 25921 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7b688e171914559b284299db6c81f3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7b688e171914559b284299db6c81f3d. Latest consensus state: current_term: 1 leader_uuid: "e7b688e171914559b284299db6c81f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ca56d74ff164da9b468da1b54080881" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 46709 } } peers { permanent_uuid: "9130c123b1054d0cac3c54319804999c" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43057 } } peers { permanent_uuid: "e7b688e171914559b284299db6c81f3d" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 33033 } } }
I20260812 16:39:45.905305 25921 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7b688e171914559b284299db6c81f3d [sys.catalog]: This master's current role is: LEADER
I20260812 16:39:45.905719 25927 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:39:45.905951 25927 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:39:45.908290 25507 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8ca56d74ff164da9b468da1b54080881 [term 1 FOLLOWER]: Refusing update from remote peer e7b688e171914559b284299db6c81f3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:39:45.908759 25921 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e7b688e171914559b284299db6c81f3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ca56d74ff164da9b468da1b54080881" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 46709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:39:45.910224 25895 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ca56d74ff164da9b468da1b54080881 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7b688e171914559b284299db6c81f3d. Latest consensus state: current_term: 1 leader_uuid: "e7b688e171914559b284299db6c81f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ca56d74ff164da9b468da1b54080881" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 46709 } } peers { permanent_uuid: "9130c123b1054d0cac3c54319804999c" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43057 } } peers { permanent_uuid: "e7b688e171914559b284299db6c81f3d" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 33033 } } }
I20260812 16:39:45.910305 25895 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ca56d74ff164da9b468da1b54080881 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:39:45.915338 25700 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.253.252" port: 33033 } attempt: 3
I20260812 16:39:45.916322 25921 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7b688e171914559b284299db6c81f3d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e7b688e171914559b284299db6c81f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ca56d74ff164da9b468da1b54080881" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 46709 } } peers { permanent_uuid: "9130c123b1054d0cac3c54319804999c" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43057 } } peers { permanent_uuid: "e7b688e171914559b284299db6c81f3d" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 33033 } } }
I20260812 16:39:45.916419 25921 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7b688e171914559b284299db6c81f3d [sys.catalog]: This master's current role is: LEADER
I20260812 16:39:45.917069 25895 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ca56d74ff164da9b468da1b54080881 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e7b688e171914559b284299db6c81f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ca56d74ff164da9b468da1b54080881" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 46709 } } peers { permanent_uuid: "9130c123b1054d0cac3c54319804999c" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43057 } } peers { permanent_uuid: "e7b688e171914559b284299db6c81f3d" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 33033 } } }
I20260812 16:39:45.917150 25895 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ca56d74ff164da9b468da1b54080881 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:39:45.917626 25929 mvcc.cc:205] Tried to move back new op lower bound from 7317720211076288512 to 7317720211063058432. Current Snapshot: MvccSnapshot[applied={T|T < 7317720211076288512}]
I20260812 16:39:45.917565 25928 mvcc.cc:205] Tried to move back new op lower bound from 7317720211076288512 to 7317720211063058432. Current Snapshot: MvccSnapshot[applied={T|T < 7317720211076288512}]
I20260812 16:39:45.918946 25927 catalog_manager.cc:1347] Generated new cluster ID: 44f54b1b8cfb464fb5567e7b2380bdf5
I20260812 16:39:45.919153 25927 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 16:39:45 dist-test-slave-w4v5 krb5kdc[25410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552785, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.253@KRBTEST.COM for kudu/127.0.253.252@KRBTEST.COM
I20260812 16:39:45.924875 25704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:45.915633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:33033 (local address 127.0.0.1:43988)
0812 16:39:45.915696 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:45.915711 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 16:39:45.915760 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:39:45.915937 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:39:45.915938 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:39:45.916136 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:39:45.916186 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:45.916188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.917137 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.917139 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:45.917953 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:45.917956 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.918476 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.918477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:45.918535 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:45.919143 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:39:45.919149 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:39:45.921811 (+ 2662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:39:45.923786 (+ 1975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:45.923787 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:45.923788 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:45.924013 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:45.924214 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:45.924215 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:45.924215 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:45.924272 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:45.924546 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:39:45.924547 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:39:45.924593 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 16:39:45.924667 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 16:39:45.924771 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 16:39:45.925230 25854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:45.915736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43988 (local address 127.0.253.252:33033)
0812 16:39:45.915777 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:45.915779 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:45.915788 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:39:45.915813 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:45.915814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:45.915824 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:45.915880 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:39:45.916272 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.917038 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.918050 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:45.918359 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:45.918417 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:45.921904 (+ 3487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:39:45.921905 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:39:45.921906 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:39:45.921913 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:39:45.923691 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:45.924091 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:45.924092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:45.924092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:45.924148 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:45.924333 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:45.924334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:45.924334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:45.924462 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:39:45.924518 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:45.925061 (+ 543us) server_negotiation.cc:299] Negotiation successful
0812 16:39:45.925143 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 16:39:45.926787 25700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9130c123b1054d0cac3c54319804999c: Bootstrap starting.
I20260812 16:39:45.927647 25700 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9130c123b1054d0cac3c54319804999c: Neither blocks nor log segments found. Creating new log.
I20260812 16:39:45.927997 25700 log.cc:824] T 00000000000000000000000000000000 P 9130c123b1054d0cac3c54319804999c: Log is configured to *not* fsync() on all Append() calls
I20260812 16:39:45.929013 25700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9130c123b1054d0cac3c54319804999c: No bootstrap required, opened a new log
I20260812 16:39:45.931135 25700 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9130c123b1054d0cac3c54319804999c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ca56d74ff164da9b468da1b54080881" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 46709 } } peers { permanent_uuid: "9130c123b1054d0cac3c54319804999c" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43057 } } peers { permanent_uuid: "e7b688e171914559b284299db6c81f3d" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 33033 } }
I20260812 16:39:45.931288 25700 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9130c123b1054d0cac3c54319804999c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:39:45.931324 25700 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9130c123b1054d0cac3c54319804999c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9130c123b1054d0cac3c54319804999c, State: Initialized, Role: FOLLOWER
I20260812 16:39:45.931443 25700 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9130c123b1054d0cac3c54319804999c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ca56d74ff164da9b468da1b54080881" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 46709 } } peers { permanent_uuid: "9130c123b1054d0cac3c54319804999c" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43057 } } peers { permanent_uuid: "e7b688e171914559b284299db6c81f3d" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 33033 } }
I20260812 16:39:45.931769 25700 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9130c123b1054d0cac3c54319804999c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:39:45.931888 25642 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9130c123b1054d0cac3c54319804999c [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:39:45.932799 25642 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9130c123b1054d0cac3c54319804999c [term 1 FOLLOWER]: Refusing update from remote peer e7b688e171914559b284299db6c81f3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:39:45.932996 25648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e7b688e171914559b284299db6c81f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9130c123b1054d0cac3c54319804999c" is_pre_election: true
I20260812 16:39:45.933110 25648 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9130c123b1054d0cac3c54319804999c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7b688e171914559b284299db6c81f3d in term 1.
I20260812 16:39:45.933173 25645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e7b688e171914559b284299db6c81f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9130c123b1054d0cac3c54319804999c"
I20260812 16:39:45.934495 25645 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9130c123b1054d0cac3c54319804999c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7b688e171914559b284299db6c81f3d in term 1.
I20260812 16:39:45.934937 25871 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e7b688e171914559b284299db6c81f3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9130c123b1054d0cac3c54319804999c" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:39:45.938015 25958 catalog_manager.cc:1259] Loaded cluster ID: 44f54b1b8cfb464fb5567e7b2380bdf5
I20260812 16:39:45.938107 25958 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9130c123b1054d0cac3c54319804999c: loading cluster ID for follower catalog manager: success
W20260812 16:39:45.938256 25958 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 9130c123b1054d0cac3c54319804999c: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260812 16:39:45.938339 25958 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:39:45.938511 25942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9130c123b1054d0cac3c54319804999c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e7b688e171914559b284299db6c81f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ca56d74ff164da9b468da1b54080881" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 46709 } } peers { permanent_uuid: "9130c123b1054d0cac3c54319804999c" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43057 } } peers { permanent_uuid: "e7b688e171914559b284299db6c81f3d" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 33033 } } }
I20260812 16:39:45.938577 25942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9130c123b1054d0cac3c54319804999c [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:39:45.938619 25942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9130c123b1054d0cac3c54319804999c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7b688e171914559b284299db6c81f3d. Latest consensus state: current_term: 1 leader_uuid: "e7b688e171914559b284299db6c81f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ca56d74ff164da9b468da1b54080881" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 46709 } } peers { permanent_uuid: "9130c123b1054d0cac3c54319804999c" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43057 } } peers { permanent_uuid: "e7b688e171914559b284299db6c81f3d" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 33033 } } }
I20260812 16:39:45.938658 25942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9130c123b1054d0cac3c54319804999c [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:39:45.938684 25942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9130c123b1054d0cac3c54319804999c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e7b688e171914559b284299db6c81f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ca56d74ff164da9b468da1b54080881" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 46709 } } peers { permanent_uuid: "9130c123b1054d0cac3c54319804999c" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43057 } } peers { permanent_uuid: "e7b688e171914559b284299db6c81f3d" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 33033 } } }
I20260812 16:39:45.938733 25942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9130c123b1054d0cac3c54319804999c [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:39:45.948630 25871 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7b688e171914559b284299db6c81f3d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e7b688e171914559b284299db6c81f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ca56d74ff164da9b468da1b54080881" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 46709 } } peers { permanent_uuid: "9130c123b1054d0cac3c54319804999c" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 43057 } } peers { permanent_uuid: "e7b688e171914559b284299db6c81f3d" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 33033 } } }
I20260812 16:39:45.948753 25871 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7b688e171914559b284299db6c81f3d [sys.catalog]: This master's current role is: LEADER
I20260812 16:39:45.950042 25927 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:39:45.950836 25927 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:39:45.954322 25927 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e7b688e171914559b284299db6c81f3d: Generated new TSK 0
I20260812 16:39:45.955123 25927 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:39:46.895207 25917 catalog_manager.cc:1259] Loaded cluster ID: 44f54b1b8cfb464fb5567e7b2380bdf5
I20260812 16:39:46.895300 25917 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8ca56d74ff164da9b468da1b54080881: loading cluster ID for follower catalog manager: success
I20260812 16:39:46.896190 25917 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8ca56d74ff164da9b468da1b54080881: acquiring CA information for follower catalog manager: success
I20260812 16:39:46.896530 25917 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8ca56d74ff164da9b468da1b54080881: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 16:39:46.939666 25958 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9130c123b1054d0cac3c54319804999c: acquiring CA information for follower catalog manager: success
I20260812 16:39:46.940093 25958 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9130c123b1054d0cac3c54319804999c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 16:39:47.015368 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 25429
W20260812 16:39:47.031962 25748 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.253.254:46709 (error 108)
I20260812 16:39:47.032390 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 25558
I20260812 16:39:47.045490 25981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:47.032643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:46709 (local address 127.0.0.1:49754)
0812 16:39:47.045318 (+ 12675us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:47.045441 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:46709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12648,"thread_start_us":51,"threads_started":1}
W20260812 16:39:47.045646 25748 consensus_peers.cc:487] T 00000000000000000000000000000000 P e7b688e171914559b284299db6c81f3d -> Peer 8ca56d74ff164da9b468da1b54080881 (127.0.253.254:46709): Couldn't send request to peer 8ca56d74ff164da9b468da1b54080881. Status: Network error: Client connection negotiation failed: client connection to 127.0.253.254:46709: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 16:39:47.051376 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 25712
2026-08-12T16:39:47Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3123 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:39:47 dist-test-slave-w4v5 krb5kdc[25986](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:39:47 dist-test-slave-w4v5 krb5kdc[25986](info): set up 2 sockets
Aug 12 16:39:47 dist-test-slave-w4v5 krb5kdc[25986](info): commencing operation
krb5kdc: starting...
W20260812 16:39:48.385495 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.304s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:39:48 dist-test-slave-w4v5 krb5kdc[25986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552788, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:39:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:39:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
I20260812 16:39:48.583636 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:35393
--webserver_interface=127.0.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:41797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:35393
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:39:48.668123 26009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:39:48.668376 26009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:39:48.668416 26009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:39:48.669595 26009 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:39:48.669663 26009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:39:48.669688 26009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:39:48.669718 26009 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:39:48.669754 26009 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:39:48.671273 26009 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:41797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:35393
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:35393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26009
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:39:48.671554 26009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:39:48.671789 26009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:39:48.676980 26022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:39:48.676957 26018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:39:48.676991 26009 server_base.cc:1034] running on GCE node
W20260812 16:39:48.676957 26019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:39:48.677470 26009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:39:48.677774 26009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:39:48.678959 26009 hybrid_clock.cc:648] HybridClock initialized: now 1786552788678946 us; error 46 us; skew 500 ppm
Aug 12 16:39:48 dist-test-slave-w4v5 krb5kdc[25986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552788, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:39:48.681529 26009 init.cc:377] Logged in from keytab as kudu/127.0.253.254@KRBTEST.COM (short username kudu)
I20260812 16:39:48.682005 26009 webserver.cc:460] Webserver started at http://127.0.253.254:44771/ using document root <none> and password file <none>
I20260812 16:39:48.682237 26009 fs_manager.cc:359] Metadata directory not provided
I20260812 16:39:48.682308 26009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:39:48.682461 26009 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:39:48.683588 26009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "6f2c4b43dc46442e85377452954a59eb"
format_stamp: "Formatted at 2026-08-12 16:39:48 on dist-test-slave-w4v5"
server_key: "2197df164940ebb48947b4ddcd815394"
server_key_iv: "01990abf5660ed4a6241b21168f72499"
server_key_version: "encryptionkey@0"
I20260812 16:39:48.683956 26009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "6f2c4b43dc46442e85377452954a59eb"
format_stamp: "Formatted at 2026-08-12 16:39:48 on dist-test-slave-w4v5"
server_key: "2197df164940ebb48947b4ddcd815394"
server_key_iv: "01990abf5660ed4a6241b21168f72499"
server_key_version: "encryptionkey@0"
I20260812 16:39:48.685326 26009 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:39:48.686043 26040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:48.686249 26009 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:39:48.686371 26009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "6f2c4b43dc46442e85377452954a59eb"
format_stamp: "Formatted at 2026-08-12 16:39:48 on dist-test-slave-w4v5"
server_key: "2197df164940ebb48947b4ddcd815394"
server_key_iv: "01990abf5660ed4a6241b21168f72499"
server_key_version: "encryptionkey@0"
I20260812 16:39:48.686446 26009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:39:48.704764 26009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:39:48.707267 26009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:39:48.707441 26009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:39:48.711879 26009 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:35393
I20260812 16:39:48.711922 26136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:35393 every 8 connection(s)
I20260812 16:39:48.712328 26009 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:39:48.713251 26137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:39:48.715602 26137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f2c4b43dc46442e85377452954a59eb: Bootstrap starting.
I20260812 16:39:48.716228 26137 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f2c4b43dc46442e85377452954a59eb: Neither blocks nor log segments found. Creating new log.
I20260812 16:39:48.716522 26137 log.cc:824] T 00000000000000000000000000000000 P 6f2c4b43dc46442e85377452954a59eb: Log is configured to *not* fsync() on all Append() calls
I20260812 16:39:48.717267 26137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f2c4b43dc46442e85377452954a59eb: No bootstrap required, opened a new log
I20260812 16:39:48.718865 26137 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f2c4b43dc46442e85377452954a59eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2c4b43dc46442e85377452954a59eb" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 35393 } }
I20260812 16:39:48.719002 26137 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f2c4b43dc46442e85377452954a59eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:39:48.719024 26137 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f2c4b43dc46442e85377452954a59eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f2c4b43dc46442e85377452954a59eb, State: Initialized, Role: FOLLOWER
I20260812 16:39:48.719238 26137 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f2c4b43dc46442e85377452954a59eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2c4b43dc46442e85377452954a59eb" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 35393 } }
I20260812 16:39:48.719359 26137 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f2c4b43dc46442e85377452954a59eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:39:48.719398 26137 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f2c4b43dc46442e85377452954a59eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:39:48.719452 26137 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f2c4b43dc46442e85377452954a59eb [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:39:48.719455 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 26009
I20260812 16:39:48.719571 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:39:48.719807 1015 external_mini_cluster.cc:1428] Setting key 0bbdf53c636ac19ea36d9ef7e7ab79be
I20260812 16:39:48.720140 26137 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f2c4b43dc46442e85377452954a59eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2c4b43dc46442e85377452954a59eb" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 35393 } }
I20260812 16:39:48.720268 26137 leader_election.cc:302] T 00000000000000000000000000000000 P 6f2c4b43dc46442e85377452954a59eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f2c4b43dc46442e85377452954a59eb; no voters:
I20260812 16:39:48.720425 26137 leader_election.cc:288] T 00000000000000000000000000000000 P 6f2c4b43dc46442e85377452954a59eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:39:48.720531 26145 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f2c4b43dc46442e85377452954a59eb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:39:48.720702 26137 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f2c4b43dc46442e85377452954a59eb [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:39:48.720726 26145 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f2c4b43dc46442e85377452954a59eb [term 1 LEADER]: Becoming Leader. State: Replica: 6f2c4b43dc46442e85377452954a59eb, State: Running, Role: LEADER
I20260812 16:39:48.720824 26145 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f2c4b43dc46442e85377452954a59eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2c4b43dc46442e85377452954a59eb" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 35393 } }
I20260812 16:39:48.721158 26145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f2c4b43dc46442e85377452954a59eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f2c4b43dc46442e85377452954a59eb. Latest consensus state: current_term: 1 leader_uuid: "6f2c4b43dc46442e85377452954a59eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2c4b43dc46442e85377452954a59eb" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 35393 } } }
I20260812 16:39:48.721256 26145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f2c4b43dc46442e85377452954a59eb [sys.catalog]: This master's current role is: LEADER
I20260812 16:39:48.721478 26155 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:39:48.721503 26147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f2c4b43dc46442e85377452954a59eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f2c4b43dc46442e85377452954a59eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2c4b43dc46442e85377452954a59eb" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 35393 } } }
I20260812 16:39:48.721647 26147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f2c4b43dc46442e85377452954a59eb [sys.catalog]: This master's current role is: LEADER
I20260812 16:39:48.722323 26155 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 16:39:48 dist-test-slave-w4v5 krb5kdc[25986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552788, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:39:48.727648 26155 catalog_manager.cc:1347] Generated new cluster ID: 41fe9ac7cb004e6ebbd180211ffaa81b
I20260812 16:39:48.727705 26155 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:39:48.732009 26144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:48.720284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52538 (local address 127.0.253.254:35393)
0812 16:39:48.720484 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:48.720495 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:48.720660 (+ 165us) server_negotiation.cc:407] Connection header received
0812 16:39:48.720800 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:48.720808 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:48.720865 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:48.721026 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:39:48.722389 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:48.723718 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:48.724502 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:48.724935 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:48.725089 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:48.727515 (+ 2426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:39:48.727520 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:39:48.727520 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:39:48.727534 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:39:48.730142 (+ 2608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:48.730593 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:48.730597 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:48.730597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:48.730727 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:48.730941 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:48.730941 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:48.730942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:48.731265 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:39:48.731431 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:48.731540 (+ 109us) server_negotiation.cc:299] Negotiation successful
0812 16:39:48.731779 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":97,"threads_started":1}
I20260812 16:39:48.732954 1015 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:39:48.762873 26155 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:39:48.763800 26155 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:39:48.767629 26155 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f2c4b43dc46442e85377452954a59eb: Generated new TSK 0
I20260812 16:39:48.767795 26155 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:39:48.786954 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 26009
2026-08-12T16:39:48Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1742 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:39:48 dist-test-slave-w4v5 krb5kdc[26187](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:39:48 dist-test-slave-w4v5 krb5kdc[26187](info): set up 2 sockets
Aug 12 16:39:48 dist-test-slave-w4v5 krb5kdc[26187](info): commencing operation
krb5kdc: starting...
W20260812 16:39:50.126397 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.303s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:39:50 dist-test-slave-w4v5 krb5kdc[26187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:39:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:39:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
I20260812 16:39:50.260347 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:41633
--webserver_interface=127.0.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:38235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:41633
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:39:50.346158 26205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:39:50.346372 26205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:39:50.346396 26205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:39:50.347589 26205 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:39:50.347631 26205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:39:50.347646 26205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:39:50.347659 26205 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:39:50.347714 26205 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:39:50.349188 26205 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:38235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:41633
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:41633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:39:50.349442 26205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:39:50.349675 26205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:39:50.354650 26214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:39:50.354691 26219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:39:50.354705 26215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:39:50.354835 26205 server_base.cc:1034] running on GCE node
I20260812 16:39:50.355157 26205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:39:50.355509 26205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:39:50.356678 26205 hybrid_clock.cc:648] HybridClock initialized: now 1786552790356656 us; error 40 us; skew 500 ppm
Aug 12 16:39:50 dist-test-slave-w4v5 krb5kdc[26187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552790, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:39:50.359333 26205 init.cc:377] Logged in from keytab as kudu/127.0.253.254@KRBTEST.COM (short username kudu)
I20260812 16:39:50.359779 26205 webserver.cc:460] Webserver started at http://127.0.253.254:33581/ using document root <none> and password file <none>
I20260812 16:39:50.360013 26205 fs_manager.cc:359] Metadata directory not provided
I20260812 16:39:50.360061 26205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:39:50.360216 26205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:39:50.361299 26205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "5c1bd1105c81469c9b3c1cf7dab17ffe"
format_stamp: "Formatted at 2026-08-12 16:39:50 on dist-test-slave-w4v5"
server_key: "b4af129b54d2289074366ee55d0dae69"
server_key_iv: "7822080e3266a7ba3984777f34d0abb6"
server_key_version: "encryptionkey@0"
I20260812 16:39:50.361680 26205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "5c1bd1105c81469c9b3c1cf7dab17ffe"
format_stamp: "Formatted at 2026-08-12 16:39:50 on dist-test-slave-w4v5"
server_key: "b4af129b54d2289074366ee55d0dae69"
server_key_iv: "7822080e3266a7ba3984777f34d0abb6"
server_key_version: "encryptionkey@0"
I20260812 16:39:50.363018 26205 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:39:50.363812 26235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:50.363991 26205 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:50.364096 26205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "5c1bd1105c81469c9b3c1cf7dab17ffe"
format_stamp: "Formatted at 2026-08-12 16:39:50 on dist-test-slave-w4v5"
server_key: "b4af129b54d2289074366ee55d0dae69"
server_key_iv: "7822080e3266a7ba3984777f34d0abb6"
server_key_version: "encryptionkey@0"
I20260812 16:39:50.364180 26205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:39:50.372179 26205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:39:50.374418 26205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:39:50.374534 26205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:39:50.378585 26205 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:41633
I20260812 16:39:50.378616 26333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:41633 every 8 connection(s)
I20260812 16:39:50.378966 26205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:39:50.380056 26334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:39:50.382411 26334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe: Bootstrap starting.
I20260812 16:39:50.383055 26334 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe: Neither blocks nor log segments found. Creating new log.
I20260812 16:39:50.383392 26334 log.cc:824] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe: Log is configured to *not* fsync() on all Append() calls
I20260812 16:39:50.384127 26334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe: No bootstrap required, opened a new log
I20260812 16:39:50.385730 26334 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1bd1105c81469c9b3c1cf7dab17ffe" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 41633 } }
I20260812 16:39:50.385896 26334 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:39:50.385936 26334 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c1bd1105c81469c9b3c1cf7dab17ffe, State: Initialized, Role: FOLLOWER
I20260812 16:39:50.386124 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 26205
I20260812 16:39:50.386122 26334 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1bd1105c81469c9b3c1cf7dab17ffe" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 41633 } }
I20260812 16:39:50.386251 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:39:50.386247 26334 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:39:50.386291 26334 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:39:50.386343 26334 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:39:50.386463 1015 external_mini_cluster.cc:1428] Setting key 9e8538b17ef802ba5e1c44cf77278443
I20260812 16:39:50.387579 26334 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1bd1105c81469c9b3c1cf7dab17ffe" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 41633 } }
I20260812 16:39:50.387694 26334 leader_election.cc:302] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c1bd1105c81469c9b3c1cf7dab17ffe; no voters:
I20260812 16:39:50.387847 26334 leader_election.cc:288] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:39:50.387868 26348 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:39:50.387981 26348 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [term 1 LEADER]: Becoming Leader. State: Replica: 5c1bd1105c81469c9b3c1cf7dab17ffe, State: Running, Role: LEADER
I20260812 16:39:50.388075 26348 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1bd1105c81469c9b3c1cf7dab17ffe" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 41633 } }
I20260812 16:39:50.388119 26334 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:39:50.388428 26352 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c1bd1105c81469c9b3c1cf7dab17ffe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1bd1105c81469c9b3c1cf7dab17ffe" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 41633 } } }
I20260812 16:39:50.388672 26352 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [sys.catalog]: This master's current role is: LEADER
I20260812 16:39:50.388585 26349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c1bd1105c81469c9b3c1cf7dab17ffe. Latest consensus state: current_term: 1 leader_uuid: "5c1bd1105c81469c9b3c1cf7dab17ffe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1bd1105c81469c9b3c1cf7dab17ffe" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 41633 } } }
I20260812 16:39:50.388926 26349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [sys.catalog]: This master's current role is: LEADER
I20260812 16:39:50.389205 26365 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:39:50.389824 26365 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:39:50.392285 26365 catalog_manager.cc:1347] Generated new cluster ID: 4f3572aa723744668178e4247dc42536
I20260812 16:39:50.392343 26365 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 16:39:50 dist-test-slave-w4v5 krb5kdc[26187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:39:50.397673 26346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:50.387003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51716 (local address 127.0.253.254:41633)
0812 16:39:50.387236 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:50.387248 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:50.387291 (+ 43us) server_negotiation.cc:407] Connection header received
0812 16:39:50.387412 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:50.387420 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:50.387479 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:50.387633 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:39:50.388457 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:50.389965 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:50.391167 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:50.391751 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:50.391857 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:50.394300 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:39:50.394305 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:39:50.394305 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:39:50.394325 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:39:50.396110 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:50.396492 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:50.396500 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:50.396501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:50.396599 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:50.396814 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:50.396815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:50.396816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:50.397058 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:39:50.397239 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:50.397327 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 16:39:50.397475 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":122,"threads_started":1}
I20260812 16:39:50.424266 26365 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:39:50.425011 26365 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:39:50.428622 26365 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe: Generated new TSK 0
I20260812 16:39:50.428859 26365 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab.
Entry for principal kudu/127.0.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab.
I20260812 16:39:50.465982 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.193:0
--local_ip_for_outbound_sockets=127.0.253.193
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:41633
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:38235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:39:50.561928 26385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:39:50.562153 26385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:39:50.562177 26385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:39:50.563537 26385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:39:50.563585 26385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:39:50.563613 26385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.193
I20260812 16:39:50.565133 26385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:38235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:41633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.193
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:39:50.565394 26385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:39:50.565665 26385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:39:50.571285 26400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:39:50.571334 26396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:39:50.571583 26385 server_base.cc:1034] running on GCE node
W20260812 16:39:50.571316 26397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:39:50.572034 26385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:39:50.572335 26385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:39:50.573535 26385 hybrid_clock.cc:648] HybridClock initialized: now 1786552790573518 us; error 35 us; skew 500 ppm
Aug 12 16:39:50 dist-test-slave-w4v5 krb5kdc[26187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552790, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:39:50.582652 26385 init.cc:377] Logged in from keytab as kudu/127.0.253.193@KRBTEST.COM (short username kudu)
I20260812 16:39:50.583254 26385 webserver.cc:460] Webserver started at http://127.0.253.193:37137/ using document root <none> and password file <none>
I20260812 16:39:50.583504 26385 fs_manager.cc:359] Metadata directory not provided
I20260812 16:39:50.583577 26385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:39:50.583731 26385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:39:50.584887 26385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/data/instance:
uuid: "9503aa07ced9420fbf374cf0b694a872"
format_stamp: "Formatted at 2026-08-12 16:39:50 on dist-test-slave-w4v5"
server_key: "2d646c30b1b6ed0c6d8463ad168ff84b"
server_key_iv: "ecadb81110145fef90035ede117b1b14"
server_key_version: "encryptionkey@0"
I20260812 16:39:50.585285 26385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance:
uuid: "9503aa07ced9420fbf374cf0b694a872"
format_stamp: "Formatted at 2026-08-12 16:39:50 on dist-test-slave-w4v5"
server_key: "2d646c30b1b6ed0c6d8463ad168ff84b"
server_key_iv: "ecadb81110145fef90035ede117b1b14"
server_key_version: "encryptionkey@0"
I20260812 16:39:50.586716 26385 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:39:50.587565 26415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:50.587852 26385 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 16:39:50.587970 26385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/wal
uuid: "9503aa07ced9420fbf374cf0b694a872"
format_stamp: "Formatted at 2026-08-12 16:39:50 on dist-test-slave-w4v5"
server_key: "2d646c30b1b6ed0c6d8463ad168ff84b"
server_key_iv: "ecadb81110145fef90035ede117b1b14"
server_key_version: "encryptionkey@0"
I20260812 16:39:50.588060 26385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:39:50.597568 26385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:39:50.599623 26385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:39:50.599743 26385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:39:50.599936 26385 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:39:50.600210 26385 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:39:50.600243 26385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:50.600265 26385 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:39:50.600278 26385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:50.606096 26385 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.193:44207
I20260812 16:39:50.606172 26632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.193:44207 every 8 connection(s)
I20260812 16:39:50.606514 26385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
Aug 12 16:39:50 dist-test-slave-w4v5 krb5kdc[26187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552790, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:39:50.613386 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 26385
I20260812 16:39:50.613530 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance
I20260812 16:39:50.613778 1015 external_mini_cluster.cc:1428] Setting key 074e461a9b9cc72647ae49873ca5d261
I20260812 16:39:50.616530 26346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:50.607350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:55351 (local address 127.0.253.254:41633)
0812 16:39:50.607429 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:50.607432 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:50.607625 (+ 193us) server_negotiation.cc:407] Connection header received
0812 16:39:50.608112 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:50.608113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:50.608128 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:50.608183 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:39:50.609112 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:50.609609 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:50.610651 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:50.610859 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:50.610939 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:50.613270 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:39:50.613272 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:39:50.613272 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:39:50.613284 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:39:50.615022 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:50.615471 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:50.615473 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:50.615473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:50.615540 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:50.615808 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:50.615809 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:50.615809 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:50.615932 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:39:50.616009 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:50.616333 (+ 324us) server_negotiation.cc:299] Negotiation successful
0812 16:39:50.616444 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 16:39:50.616761 26638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:50.607299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.193:55351)
0812 16:39:50.607493 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:50.607532 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 16:39:50.608020 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:39:50.608238 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:39:50.608241 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:39:50.608822 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:39:50.609016 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:50.609021 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:50.609719 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:50.609720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:50.610544 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:50.610546 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:50.610959 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:50.610960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:50.611124 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:50.611689 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:39:50.611698 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:39:50.613161 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:39:50.615142 (+ 1981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:50.615150 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:50.615152 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:50.615381 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:50.615612 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:50.615613 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:50.615614 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:50.615732 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:50.616000 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:39:50.616008 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:39:50.616152 (+ 144us) client_negotiation.cc:769] Sending connection context
0812 16:39:50.616274 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 16:39:50.616478 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":103,"threads_started":1}
I20260812 16:39:50.617425 26633 heartbeater.cc:347] Connected to a master server at 127.0.253.254:41633
I20260812 16:39:50.617552 26633 heartbeater.cc:464] Registering TS with master...
I20260812 16:39:50.617744 26633 heartbeater.cc:511] Master 127.0.253.254:41633 requested a full tablet report, sending...
I20260812 16:39:50.618395 26265 ts_manager.cc:194] Registered new tserver with Master: 9503aa07ced9420fbf374cf0b694a872 (127.0.253.193:44207)
I20260812 16:39:50.619390 26265 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.193@KRBTEST.COM'} at 127.0.253.193:55351
WARNING: no policy specified for kudu/127.0.253.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab.
Entry for principal kudu/127.0.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab.
I20260812 16:39:50.644212 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.194:0
--local_ip_for_outbound_sockets=127.0.253.194
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:41633
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:38235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:39:50.727885 26644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:39:50.728166 26644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:39:50.728209 26644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:39:50.729393 26644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:39:50.729468 26644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:39:50.729511 26644 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.194
I20260812 16:39:50.731117 26644 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:38235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:41633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.194
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:39:50.731405 26644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:39:50.731662 26644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:39:50.737036 26660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:39:50.737027 26653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:39:50.737036 26652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:39:50.737361 26644 server_base.cc:1034] running on GCE node
I20260812 16:39:50.737599 26644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:39:50.737818 26644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:39:50.739012 26644 hybrid_clock.cc:648] HybridClock initialized: now 1786552790738997 us; error 26 us; skew 500 ppm
Aug 12 16:39:50 dist-test-slave-w4v5 krb5kdc[26187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552790, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:39:50.741768 26644 init.cc:377] Logged in from keytab as kudu/127.0.253.194@KRBTEST.COM (short username kudu)
I20260812 16:39:50.742220 26644 webserver.cc:460] Webserver started at http://127.0.253.194:42105/ using document root <none> and password file <none>
I20260812 16:39:50.742482 26644 fs_manager.cc:359] Metadata directory not provided
I20260812 16:39:50.742563 26644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:39:50.742714 26644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:39:50.743847 26644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/data/instance:
uuid: "2bcb06bb050f462bbe02533992ffb681"
format_stamp: "Formatted at 2026-08-12 16:39:50 on dist-test-slave-w4v5"
server_key: "f334eca05f1bda10a835ce781e2d5e46"
server_key_iv: "a369b104688bd16f17ffd88c8149316d"
server_key_version: "encryptionkey@0"
I20260812 16:39:50.744215 26644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance:
uuid: "2bcb06bb050f462bbe02533992ffb681"
format_stamp: "Formatted at 2026-08-12 16:39:50 on dist-test-slave-w4v5"
server_key: "f334eca05f1bda10a835ce781e2d5e46"
server_key_iv: "a369b104688bd16f17ffd88c8149316d"
server_key_version: "encryptionkey@0"
I20260812 16:39:50.745656 26644 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:39:50.746587 26672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:50.746783 26644 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:39:50.746896 26644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/wal
uuid: "2bcb06bb050f462bbe02533992ffb681"
format_stamp: "Formatted at 2026-08-12 16:39:50 on dist-test-slave-w4v5"
server_key: "f334eca05f1bda10a835ce781e2d5e46"
server_key_iv: "a369b104688bd16f17ffd88c8149316d"
server_key_version: "encryptionkey@0"
I20260812 16:39:50.746973 26644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:39:50.759333 26644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:39:50.762351 26644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:39:50.762519 26644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:39:50.762810 26644 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:39:50.763255 26644 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:39:50.763309 26644 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:50.763341 26644 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:39:50.763363 26644 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:50.769909 26644 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.194:38999
I20260812 16:39:50.770030 26888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.194:38999 every 8 connection(s)
I20260812 16:39:50.770336 26644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
Aug 12 16:39:50 dist-test-slave-w4v5 krb5kdc[26187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552790, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:39:50.780077 26346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:50.771193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:53315 (local address 127.0.253.254:41633)
0812 16:39:50.771269 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:50.771272 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:50.771483 (+ 211us) server_negotiation.cc:407] Connection header received
0812 16:39:50.772084 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:50.772085 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:50.772099 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:50.772167 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:39:50.773201 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:50.773849 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:50.774879 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:50.775146 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:50.775232 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:50.777500 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:39:50.777502 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:39:50.777502 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:39:50.777509 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:39:50.778774 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:50.779208 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:50.779208 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:50.779209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:50.779261 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:50.779479 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:50.779479 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:50.779479 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:50.779610 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:39:50.779664 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:50.779936 (+ 272us) server_negotiation.cc:299] Negotiation successful
0812 16:39:50.780007 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 16:39:50.780328 26897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:50.771176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.194:53315)
0812 16:39:50.771351 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:50.771391 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 16:39:50.771955 (+ 564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:39:50.772241 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:39:50.772245 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:39:50.772862 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:39:50.773094 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:50.773099 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:50.773949 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:50.773951 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:50.774783 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:50.774786 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:50.775238 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:50.775239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:50.775375 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:50.775884 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:39:50.775892 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:39:50.777408 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:39:50.778855 (+ 1447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:50.778861 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:50.778863 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:50.779133 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:50.779321 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:50.779322 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:50.779322 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:50.779425 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:50.779690 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:39:50.779694 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:39:50.779813 (+ 119us) client_negotiation.cc:769] Sending connection context
0812 16:39:50.779899 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 16:39:50.780025 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":73,"threads_started":1}
I20260812 16:39:50.780311 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 26644
I20260812 16:39:50.780412 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance
I20260812 16:39:50.780656 1015 external_mini_cluster.cc:1428] Setting key d91ec68a7531f03a821fe4523407746c
I20260812 16:39:50.780881 26890 heartbeater.cc:347] Connected to a master server at 127.0.253.254:41633
I20260812 16:39:50.780998 26890 heartbeater.cc:464] Registering TS with master...
I20260812 16:39:50.781189 26890 heartbeater.cc:511] Master 127.0.253.254:41633 requested a full tablet report, sending...
I20260812 16:39:50.781600 26265 ts_manager.cc:194] Registered new tserver with Master: 2bcb06bb050f462bbe02533992ffb681 (127.0.253.194:38999)
I20260812 16:39:50.782143 26265 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.194@KRBTEST.COM'} at 127.0.253.194:53315
WARNING: no policy specified for kudu/127.0.253.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab.
Entry for principal kudu/127.0.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab.
I20260812 16:39:50.810245 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.195:0
--local_ip_for_outbound_sockets=127.0.253.195
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:41633
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:38235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:39:50.894598 26900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:39:50.894809 26900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:39:50.894832 26900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:39:50.895975 26900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:39:50.896019 26900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:39:50.896045 26900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.195
I20260812 16:39:50.897524 26900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:38235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:41633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.195
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:39:50.897809 26900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:39:50.898051 26900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:39:50.903254 26911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:39:50.903273 26916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:39:50.903266 26908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:39:50.903514 26900 server_base.cc:1034] running on GCE node
I20260812 16:39:50.903739 26900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:39:50.903992 26900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:39:50.905200 26900 hybrid_clock.cc:648] HybridClock initialized: now 1786552790905179 us; error 29 us; skew 500 ppm
Aug 12 16:39:50 dist-test-slave-w4v5 krb5kdc[26187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552790, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:39:50.908118 26900 init.cc:377] Logged in from keytab as kudu/127.0.253.195@KRBTEST.COM (short username kudu)
I20260812 16:39:50.908600 26900 webserver.cc:460] Webserver started at http://127.0.253.195:45895/ using document root <none> and password file <none>
I20260812 16:39:50.908875 26900 fs_manager.cc:359] Metadata directory not provided
I20260812 16:39:50.908950 26900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:39:50.909106 26900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:39:50.910194 26900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/data/instance:
uuid: "1beb07e0229c436dba3ca8ab225dcbc2"
format_stamp: "Formatted at 2026-08-12 16:39:50 on dist-test-slave-w4v5"
server_key: "2eb8de141d7e4d20cd6f3732316a1be2"
server_key_iv: "cc52484f739097cf29b3d5e1a3d4517b"
server_key_version: "encryptionkey@0"
I20260812 16:39:50.910576 26900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance:
uuid: "1beb07e0229c436dba3ca8ab225dcbc2"
format_stamp: "Formatted at 2026-08-12 16:39:50 on dist-test-slave-w4v5"
server_key: "2eb8de141d7e4d20cd6f3732316a1be2"
server_key_iv: "cc52484f739097cf29b3d5e1a3d4517b"
server_key_version: "encryptionkey@0"
I20260812 16:39:50.912135 26900 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:39:50.913049 26932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:50.913266 26900 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:39:50.913394 26900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/wal
uuid: "1beb07e0229c436dba3ca8ab225dcbc2"
format_stamp: "Formatted at 2026-08-12 16:39:50 on dist-test-slave-w4v5"
server_key: "2eb8de141d7e4d20cd6f3732316a1be2"
server_key_iv: "cc52484f739097cf29b3d5e1a3d4517b"
server_key_version: "encryptionkey@0"
I20260812 16:39:50.913475 26900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:39:50.927466 26900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:39:50.929930 26900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:39:50.930105 26900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:39:50.930359 26900 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:39:50.930696 26900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:39:50.930754 26900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:50.930799 26900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:39:50.930827 26900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:50.936964 26900 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.195:36289
I20260812 16:39:50.937004 27137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.195:36289 every 8 connection(s)
I20260812 16:39:50.937384 26900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
Aug 12 16:39:50 dist-test-slave-w4v5 krb5kdc[26187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552790, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:39:50.946446 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 26900
I20260812 16:39:50.946548 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance
I20260812 16:39:50.946751 1015 external_mini_cluster.cc:1428] Setting key 0492f43e3754670ae7451d181b4031c8
I20260812 16:39:50.946873 26346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:50.938195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:59825 (local address 127.0.253.254:41633)
0812 16:39:50.938263 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:50.938265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:50.938447 (+ 182us) server_negotiation.cc:407] Connection header received
0812 16:39:50.938934 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:50.938935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:50.938946 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:50.939010 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:39:50.940003 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:50.940514 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:50.941503 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:50.941718 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:50.941775 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:50.944012 (+ 2237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:39:50.944014 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:39:50.944014 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:39:50.944021 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:39:50.945285 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:50.945676 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:50.945677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:50.945677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:50.945730 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:50.945984 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:50.945985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:50.945985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:50.946094 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:39:50.946149 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:50.946681 (+ 532us) server_negotiation.cc:299] Negotiation successful
0812 16:39:50.946756 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 16:39:50.947150 27143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:50.938156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.195:59825)
0812 16:39:50.938316 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:50.938358 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 16:39:50.938839 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:39:50.939063 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:39:50.939066 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:39:50.939702 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:39:50.939903 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:50.939907 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:50.940633 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:50.940634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:50.941423 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:50.941425 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:50.941795 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:50.941796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:50.941901 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:50.942447 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:39:50.942455 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:39:50.943922 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:39:50.945366 (+ 1444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:50.945372 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:50.945374 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:50.945606 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:50.945810 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:50.945811 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:50.945811 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:50.945921 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:50.946281 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:39:50.946288 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:39:50.946458 (+ 170us) client_negotiation.cc:769] Sending connection context
0812 16:39:50.946587 (+ 129us) client_negotiation.cc:240] Negotiation successful
0812 16:39:50.946779 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":87,"threads_started":1}
I20260812 16:39:50.947642 27139 heartbeater.cc:347] Connected to a master server at 127.0.253.254:41633
I20260812 16:39:50.947744 27139 heartbeater.cc:464] Registering TS with master...
I20260812 16:39:50.947938 27139 heartbeater.cc:511] Master 127.0.253.254:41633 requested a full tablet report, sending...
I20260812 16:39:50.948297 26265 ts_manager.cc:194] Registered new tserver with Master: 1beb07e0229c436dba3ca8ab225dcbc2 (127.0.253.195:36289)
I20260812 16:39:50.948640 26265 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.195@KRBTEST.COM'} at 127.0.253.195:59825
I20260812 16:39:50.958907 1015 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:39:50.966545 26346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:50.960369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51724 (local address 127.0.253.254:41633)
0812 16:39:50.960455 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:50.960458 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:50.960531 (+ 73us) server_negotiation.cc:407] Connection header received
0812 16:39:50.960616 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:50.960617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:50.960628 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:50.960696 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:39:50.961283 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:50.961771 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:50.962507 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:50.962702 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:50.962776 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:50.964102 (+ 1326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:39:50.964103 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:39:50.964104 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:39:50.964112 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:39:50.965327 (+ 1215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:50.965706 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:50.965708 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:50.965708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:50.965769 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:50.965970 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:50.965971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:50.965972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:50.966131 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:39:50.966244 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:50.966356 (+ 112us) server_negotiation.cc:299] Negotiation successful
0812 16:39:50.966442 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 16:39:50.968219 26265 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51724:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:39:50.969760 26265 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:39:50.976195 27163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:50.972092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:44207 (local address 127.0.0.1:50498)
0812 16:39:50.972278 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:50.972319 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 16:39:50.972389 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:39:50.972693 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:39:50.972697 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:39:50.972700 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:39:50.972770 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:50.972772 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:50.973838 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:50.973839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:50.975005 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:50.975009 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:50.975856 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:50.975857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:50.975946 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:50.975959 (+ 13us) client_negotiation.cc:769] Sending connection context
0812 16:39:50.976064 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 16:39:50.976120 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":76,"threads_started":1}
I20260812 16:39:50.976212 27170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:50.972303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50498 (local address 127.0.253.193:44207)
0812 16:39:50.972508 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:50.972518 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:50.972527 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:39:50.972558 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:50.972560 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:50.972573 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:50.972683 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:39:50.972855 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:50.973703 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:50.975122 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:50.975770 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:50.975856 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:50.975917 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:50.976083 (+ 166us) server_negotiation.cc:299] Negotiation successful
0812 16:39:50.976124 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":87,"threads_started":1}
I20260812 16:39:50.976629 27166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:50.972111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:38999 (local address 127.0.0.1:58440)
0812 16:39:50.972701 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:50.972710 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 16:39:50.972755 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:39:50.972948 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:39:50.972948 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:39:50.972951 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:39:50.973005 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:50.973007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:50.974317 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:50.974318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:50.975458 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:50.975461 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:50.976333 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:50.976334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:50.976405 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:50.976411 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 16:39:50.976471 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 16:39:50.976519 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"mutex_wait_us":26,"thread_start_us":114,"threads_started":1}
I20260812 16:39:50.976629 27168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:50.972093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.195:36289 (local address 127.0.0.1:38644)
0812 16:39:50.972545 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:50.972556 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:39:50.972621 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:39:50.973092 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:39:50.973092 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:39:50.973095 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:39:50.973135 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:50.973137 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:50.974371 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:50.974372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:50.975360 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:50.975362 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:50.976335 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:50.976335 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:50.976405 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:50.976413 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 16:39:50.976474 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 16:39:50.976520 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"mutex_wait_us":13,"thread_start_us":86,"threads_started":1}
I20260812 16:39:50.976656 27169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:50.972259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38644 (local address 127.0.253.195:36289)
0812 16:39:50.972484 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:50.972489 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:50.972614 (+ 125us) server_negotiation.cc:407] Connection header received
0812 16:39:50.972711 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:50.972714 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:50.972734 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:50.972813 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:39:50.973219 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:50.974284 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:50.975458 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:50.976157 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:50.976235 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:50.976293 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:50.976506 (+ 213us) server_negotiation.cc:299] Negotiation successful
0812 16:39:50.976556 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":85,"threads_started":1}
I20260812 16:39:50.977207 27172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:50.972380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58440 (local address 127.0.253.194:38999)
0812 16:39:50.972562 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:50.972565 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:50.972817 (+ 252us) server_negotiation.cc:407] Connection header received
0812 16:39:50.972861 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:50.972864 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:50.972880 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:50.972950 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:39:50.973062 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:50.974212 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:50.975525 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:50.976193 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:50.976250 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:50.976294 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:50.977074 (+ 780us) server_negotiation.cc:299] Negotiation successful
0812 16:39:50.977117 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":81,"threads_started":1}
I20260812 16:39:50.977253 26511 tablet_service.cc:1467] Processing CreateTablet for tablet 94b6c7a35ed247b29fb9cb779d051f1b (DEFAULT_TABLE table=test-table [id=ca8aca19327f428993b54609993be76d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:39:50.977475 27028 tablet_service.cc:1467] Processing CreateTablet for tablet 94b6c7a35ed247b29fb9cb779d051f1b (DEFAULT_TABLE table=test-table [id=ca8aca19327f428993b54609993be76d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:39:50.977639 26511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94b6c7a35ed247b29fb9cb779d051f1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:39:50.977804 27028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94b6c7a35ed247b29fb9cb779d051f1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:39:50.977845 26760 tablet_service.cc:1467] Processing CreateTablet for tablet 94b6c7a35ed247b29fb9cb779d051f1b (DEFAULT_TABLE table=test-table [id=ca8aca19327f428993b54609993be76d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:39:50.978144 26760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94b6c7a35ed247b29fb9cb779d051f1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:39:50.980221 27173 tablet_bootstrap.cc:492] T 94b6c7a35ed247b29fb9cb779d051f1b P 9503aa07ced9420fbf374cf0b694a872: Bootstrap starting.
I20260812 16:39:50.980937 27173 tablet_bootstrap.cc:654] T 94b6c7a35ed247b29fb9cb779d051f1b P 9503aa07ced9420fbf374cf0b694a872: Neither blocks nor log segments found. Creating new log.
I20260812 16:39:50.981412 27178 tablet_bootstrap.cc:492] T 94b6c7a35ed247b29fb9cb779d051f1b P 2bcb06bb050f462bbe02533992ffb681: Bootstrap starting.
I20260812 16:39:50.981539 27173 log.cc:824] T 94b6c7a35ed247b29fb9cb779d051f1b P 9503aa07ced9420fbf374cf0b694a872: Log is configured to *not* fsync() on all Append() calls
I20260812 16:39:50.981634 27174 tablet_bootstrap.cc:492] T 94b6c7a35ed247b29fb9cb779d051f1b P 1beb07e0229c436dba3ca8ab225dcbc2: Bootstrap starting.
I20260812 16:39:50.982113 27178 tablet_bootstrap.cc:654] T 94b6c7a35ed247b29fb9cb779d051f1b P 2bcb06bb050f462bbe02533992ffb681: Neither blocks nor log segments found. Creating new log.
I20260812 16:39:50.982440 27173 tablet_bootstrap.cc:492] T 94b6c7a35ed247b29fb9cb779d051f1b P 9503aa07ced9420fbf374cf0b694a872: No bootstrap required, opened a new log
I20260812 16:39:50.982451 27174 tablet_bootstrap.cc:654] T 94b6c7a35ed247b29fb9cb779d051f1b P 1beb07e0229c436dba3ca8ab225dcbc2: Neither blocks nor log segments found. Creating new log.
I20260812 16:39:50.982528 27173 ts_tablet_manager.cc:1397] T 94b6c7a35ed247b29fb9cb779d051f1b P 9503aa07ced9420fbf374cf0b694a872: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:39:50.982748 27178 log.cc:824] T 94b6c7a35ed247b29fb9cb779d051f1b P 2bcb06bb050f462bbe02533992ffb681: Log is configured to *not* fsync() on all Append() calls
I20260812 16:39:50.983135 27174 log.cc:824] T 94b6c7a35ed247b29fb9cb779d051f1b P 1beb07e0229c436dba3ca8ab225dcbc2: Log is configured to *not* fsync() on all Append() calls
I20260812 16:39:50.983599 27178 tablet_bootstrap.cc:492] T 94b6c7a35ed247b29fb9cb779d051f1b P 2bcb06bb050f462bbe02533992ffb681: No bootstrap required, opened a new log
I20260812 16:39:50.983716 27178 ts_tablet_manager.cc:1397] T 94b6c7a35ed247b29fb9cb779d051f1b P 2bcb06bb050f462bbe02533992ffb681: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:39:50.983899 27174 tablet_bootstrap.cc:492] T 94b6c7a35ed247b29fb9cb779d051f1b P 1beb07e0229c436dba3ca8ab225dcbc2: No bootstrap required, opened a new log
I20260812 16:39:50.983955 27174 ts_tablet_manager.cc:1397] T 94b6c7a35ed247b29fb9cb779d051f1b P 1beb07e0229c436dba3ca8ab225dcbc2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 16:39:50.984735 27173 raft_consensus.cc:348] T 94b6c7a35ed247b29fb9cb779d051f1b P 9503aa07ced9420fbf374cf0b694a872 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beb07e0229c436dba3ca8ab225dcbc2" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 36289 } } peers { permanent_uuid: "9503aa07ced9420fbf374cf0b694a872" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 44207 } } peers { permanent_uuid: "2bcb06bb050f462bbe02533992ffb681" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38999 } }
I20260812 16:39:50.984927 27173 raft_consensus.cc:374] T 94b6c7a35ed247b29fb9cb779d051f1b P 9503aa07ced9420fbf374cf0b694a872 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:39:50.984965 27173 raft_consensus.cc:729] T 94b6c7a35ed247b29fb9cb779d051f1b P 9503aa07ced9420fbf374cf0b694a872 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9503aa07ced9420fbf374cf0b694a872, State: Initialized, Role: FOLLOWER
I20260812 16:39:50.985107 27173 consensus_queue.cc:260] T 94b6c7a35ed247b29fb9cb779d051f1b P 9503aa07ced9420fbf374cf0b694a872 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beb07e0229c436dba3ca8ab225dcbc2" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 36289 } } peers { permanent_uuid: "9503aa07ced9420fbf374cf0b694a872" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 44207 } } peers { permanent_uuid: "2bcb06bb050f462bbe02533992ffb681" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38999 } }
I20260812 16:39:50.985391 27173 ts_tablet_manager.cc:1428] T 94b6c7a35ed247b29fb9cb779d051f1b P 9503aa07ced9420fbf374cf0b694a872: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:39:50.985549 26633 heartbeater.cc:503] Master 127.0.253.254:41633 was elected leader, sending a full tablet report...
I20260812 16:39:50.985800 27178 raft_consensus.cc:348] T 94b6c7a35ed247b29fb9cb779d051f1b P 2bcb06bb050f462bbe02533992ffb681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beb07e0229c436dba3ca8ab225dcbc2" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 36289 } } peers { permanent_uuid: "9503aa07ced9420fbf374cf0b694a872" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 44207 } } peers { permanent_uuid: "2bcb06bb050f462bbe02533992ffb681" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38999 } }
I20260812 16:39:50.985913 27178 raft_consensus.cc:374] T 94b6c7a35ed247b29fb9cb779d051f1b P 2bcb06bb050f462bbe02533992ffb681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:39:50.985771 27174 raft_consensus.cc:348] T 94b6c7a35ed247b29fb9cb779d051f1b P 1beb07e0229c436dba3ca8ab225dcbc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beb07e0229c436dba3ca8ab225dcbc2" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 36289 } } peers { permanent_uuid: "9503aa07ced9420fbf374cf0b694a872" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 44207 } } peers { permanent_uuid: "2bcb06bb050f462bbe02533992ffb681" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38999 } }
I20260812 16:39:50.985931 27178 raft_consensus.cc:729] T 94b6c7a35ed247b29fb9cb779d051f1b P 2bcb06bb050f462bbe02533992ffb681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bcb06bb050f462bbe02533992ffb681, State: Initialized, Role: FOLLOWER
I20260812 16:39:50.985944 27174 raft_consensus.cc:374] T 94b6c7a35ed247b29fb9cb779d051f1b P 1beb07e0229c436dba3ca8ab225dcbc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:39:50.985985 27174 raft_consensus.cc:729] T 94b6c7a35ed247b29fb9cb779d051f1b P 1beb07e0229c436dba3ca8ab225dcbc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1beb07e0229c436dba3ca8ab225dcbc2, State: Initialized, Role: FOLLOWER
I20260812 16:39:50.986040 27178 consensus_queue.cc:260] T 94b6c7a35ed247b29fb9cb779d051f1b P 2bcb06bb050f462bbe02533992ffb681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beb07e0229c436dba3ca8ab225dcbc2" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 36289 } } peers { permanent_uuid: "9503aa07ced9420fbf374cf0b694a872" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 44207 } } peers { permanent_uuid: "2bcb06bb050f462bbe02533992ffb681" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38999 } }
I20260812 16:39:50.986093 27174 consensus_queue.cc:260] T 94b6c7a35ed247b29fb9cb779d051f1b P 1beb07e0229c436dba3ca8ab225dcbc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beb07e0229c436dba3ca8ab225dcbc2" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 36289 } } peers { permanent_uuid: "9503aa07ced9420fbf374cf0b694a872" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 44207 } } peers { permanent_uuid: "2bcb06bb050f462bbe02533992ffb681" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38999 } }
I20260812 16:39:50.986368 27174 ts_tablet_manager.cc:1428] T 94b6c7a35ed247b29fb9cb779d051f1b P 1beb07e0229c436dba3ca8ab225dcbc2: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260812 16:39:50.986352 27178 ts_tablet_manager.cc:1428] T 94b6c7a35ed247b29fb9cb779d051f1b P 2bcb06bb050f462bbe02533992ffb681: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 16:39:50.986352 26890 heartbeater.cc:503] Master 127.0.253.254:41633 was elected leader, sending a full tablet report...
I20260812 16:39:50.986624 27139 heartbeater.cc:503] Master 127.0.253.254:41633 was elected leader, sending a full tablet report...
W20260812 16:39:51.020973 26891 tablet.cc:2369] T 94b6c7a35ed247b29fb9cb779d051f1b P 2bcb06bb050f462bbe02533992ffb681: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 16:39:51.107405 26636 tablet.cc:2369] T 94b6c7a35ed247b29fb9cb779d051f1b P 9503aa07ced9420fbf374cf0b694a872: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 16:39:51.188016 27141 tablet.cc:2369] T 94b6c7a35ed247b29fb9cb779d051f1b P 1beb07e0229c436dba3ca8ab225dcbc2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:39:51.224298 27192 raft_consensus.cc:482] T 94b6c7a35ed247b29fb9cb779d051f1b P 1beb07e0229c436dba3ca8ab225dcbc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:39:51.224442 27192 raft_consensus.cc:504] T 94b6c7a35ed247b29fb9cb779d051f1b P 1beb07e0229c436dba3ca8ab225dcbc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beb07e0229c436dba3ca8ab225dcbc2" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 36289 } } peers { permanent_uuid: "9503aa07ced9420fbf374cf0b694a872" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 44207 } } peers { permanent_uuid: "2bcb06bb050f462bbe02533992ffb681" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38999 } }
I20260812 16:39:51.224947 27192 leader_election.cc:288] T 94b6c7a35ed247b29fb9cb779d051f1b P 1beb07e0229c436dba3ca8ab225dcbc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9503aa07ced9420fbf374cf0b694a872 (127.0.253.193:44207), 2bcb06bb050f462bbe02533992ffb681 (127.0.253.194:38999)
I20260812 16:39:51.228333 27143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:51.225159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:44207 (local address 127.0.253.195:50555)
0812 16:39:51.225222 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:51.225234 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:39:51.225281 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:39:51.225474 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:39:51.225475 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:39:51.225477 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:39:51.225588 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:51.225591 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.226574 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.226575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:51.227416 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:51.227419 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.228046 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.228047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:51.228133 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:51.228138 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:39:51.228213 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 16:39:51.228258 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 16:39:51.228363 27170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:51.225248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:50555 (local address 127.0.253.193:44207)
0812 16:39:51.225299 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:51.225301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:51.225331 (+ 30us) server_negotiation.cc:407] Connection header received
0812 16:39:51.225366 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:51.225367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:51.225378 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:51.225442 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:39:51.225678 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.226473 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.227532 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.227962 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.228020 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:51.228027 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:51.228229 (+ 202us) server_negotiation.cc:299] Negotiation successful
0812 16:39:51.228275 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 16:39:51.228457 27195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:51.225200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:38999 (local address 127.0.253.195:50331)
0812 16:39:51.225392 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:51.225399 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0812 16:39:51.225461 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:39:51.225619 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:39:51.225620 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:39:51.225622 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:39:51.225738 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:51.225740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.226607 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.226609 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:51.227576 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:51.227579 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.228183 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.228184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:51.228274 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:51.228278 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:39:51.228337 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 16:39:51.228373 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":3,"thread_start_us":135,"threads_started":1}
I20260812 16:39:51.228493 27172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:51.225268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:50331 (local address 127.0.253.194:38999)
0812 16:39:51.225335 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:51.225337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:51.225460 (+ 123us) server_negotiation.cc:407] Connection header received
0812 16:39:51.225519 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:51.225521 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:51.225531 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:51.225602 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:39:51.225842 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.226525 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.227674 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.228096 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.228172 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:51.228181 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:51.228362 (+ 181us) server_negotiation.cc:299] Negotiation successful
0812 16:39:51.228413 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 16:39:51.228757 26546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94b6c7a35ed247b29fb9cb779d051f1b" candidate_uuid: "1beb07e0229c436dba3ca8ab225dcbc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9503aa07ced9420fbf374cf0b694a872" is_pre_election: true
I20260812 16:39:51.228806 26800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94b6c7a35ed247b29fb9cb779d051f1b" candidate_uuid: "1beb07e0229c436dba3ca8ab225dcbc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bcb06bb050f462bbe02533992ffb681" is_pre_election: true
I20260812 16:39:51.228943 26800 raft_consensus.cc:2454] T 94b6c7a35ed247b29fb9cb779d051f1b P 2bcb06bb050f462bbe02533992ffb681 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1beb07e0229c436dba3ca8ab225dcbc2 in term 0.
I20260812 16:39:51.228943 26546 raft_consensus.cc:2454] T 94b6c7a35ed247b29fb9cb779d051f1b P 9503aa07ced9420fbf374cf0b694a872 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1beb07e0229c436dba3ca8ab225dcbc2 in term 0.
I20260812 16:39:51.229188 26936 leader_election.cc:302] T 94b6c7a35ed247b29fb9cb779d051f1b P 1beb07e0229c436dba3ca8ab225dcbc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1beb07e0229c436dba3ca8ab225dcbc2, 2bcb06bb050f462bbe02533992ffb681; no voters:
I20260812 16:39:51.229403 27192 raft_consensus.cc:2781] T 94b6c7a35ed247b29fb9cb779d051f1b P 1beb07e0229c436dba3ca8ab225dcbc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:39:51.229465 27192 raft_consensus.cc:482] T 94b6c7a35ed247b29fb9cb779d051f1b P 1beb07e0229c436dba3ca8ab225dcbc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:39:51.229481 27192 raft_consensus.cc:3037] T 94b6c7a35ed247b29fb9cb779d051f1b P 1beb07e0229c436dba3ca8ab225dcbc2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:39:51.230258 27192 raft_consensus.cc:504] T 94b6c7a35ed247b29fb9cb779d051f1b P 1beb07e0229c436dba3ca8ab225dcbc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beb07e0229c436dba3ca8ab225dcbc2" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 36289 } } peers { permanent_uuid: "9503aa07ced9420fbf374cf0b694a872" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 44207 } } peers { permanent_uuid: "2bcb06bb050f462bbe02533992ffb681" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38999 } }
I20260812 16:39:51.230473 27192 leader_election.cc:288] T 94b6c7a35ed247b29fb9cb779d051f1b P 1beb07e0229c436dba3ca8ab225dcbc2 [CANDIDATE]: Term 1 election: Requested vote from peers 9503aa07ced9420fbf374cf0b694a872 (127.0.253.193:44207), 2bcb06bb050f462bbe02533992ffb681 (127.0.253.194:38999)
I20260812 16:39:51.230615 26800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94b6c7a35ed247b29fb9cb779d051f1b" candidate_uuid: "1beb07e0229c436dba3ca8ab225dcbc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bcb06bb050f462bbe02533992ffb681"
I20260812 16:39:51.230716 26800 raft_consensus.cc:3037] T 94b6c7a35ed247b29fb9cb779d051f1b P 2bcb06bb050f462bbe02533992ffb681 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:39:51.230728 26546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94b6c7a35ed247b29fb9cb779d051f1b" candidate_uuid: "1beb07e0229c436dba3ca8ab225dcbc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9503aa07ced9420fbf374cf0b694a872"
I20260812 16:39:51.230815 26546 raft_consensus.cc:3037] T 94b6c7a35ed247b29fb9cb779d051f1b P 9503aa07ced9420fbf374cf0b694a872 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:39:51.231520 26800 raft_consensus.cc:2454] T 94b6c7a35ed247b29fb9cb779d051f1b P 2bcb06bb050f462bbe02533992ffb681 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1beb07e0229c436dba3ca8ab225dcbc2 in term 1.
I20260812 16:39:51.231631 26546 raft_consensus.cc:2454] T 94b6c7a35ed247b29fb9cb779d051f1b P 9503aa07ced9420fbf374cf0b694a872 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1beb07e0229c436dba3ca8ab225dcbc2 in term 1.
I20260812 16:39:51.231746 26936 leader_election.cc:302] T 94b6c7a35ed247b29fb9cb779d051f1b P 1beb07e0229c436dba3ca8ab225dcbc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1beb07e0229c436dba3ca8ab225dcbc2, 2bcb06bb050f462bbe02533992ffb681; no voters:
I20260812 16:39:51.231819 27192 raft_consensus.cc:2781] T 94b6c7a35ed247b29fb9cb779d051f1b P 1beb07e0229c436dba3ca8ab225dcbc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:39:51.231992 27192 raft_consensus.cc:686] T 94b6c7a35ed247b29fb9cb779d051f1b P 1beb07e0229c436dba3ca8ab225dcbc2 [term 1 LEADER]: Becoming Leader. State: Replica: 1beb07e0229c436dba3ca8ab225dcbc2, State: Running, Role: LEADER
I20260812 16:39:51.232105 27192 consensus_queue.cc:237] T 94b6c7a35ed247b29fb9cb779d051f1b P 1beb07e0229c436dba3ca8ab225dcbc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beb07e0229c436dba3ca8ab225dcbc2" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 36289 } } peers { permanent_uuid: "9503aa07ced9420fbf374cf0b694a872" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 44207 } } peers { permanent_uuid: "2bcb06bb050f462bbe02533992ffb681" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38999 } }
I20260812 16:39:51.233078 26264 catalog_manager.cc:5495] T 94b6c7a35ed247b29fb9cb779d051f1b P 1beb07e0229c436dba3ca8ab225dcbc2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1beb07e0229c436dba3ca8ab225dcbc2 (127.0.253.195). New cstate: current_term: 1 leader_uuid: "1beb07e0229c436dba3ca8ab225dcbc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beb07e0229c436dba3ca8ab225dcbc2" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 36289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9503aa07ced9420fbf374cf0b694a872" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 44207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2bcb06bb050f462bbe02533992ffb681" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38999 } health_report { overall_health: UNKNOWN } } }
I20260812 16:39:51.264042 27169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:51.261518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38656 (local address 127.0.253.195:36289)
0812 16:39:51.261582 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:51.261584 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:51.261726 (+ 142us) server_negotiation.cc:407] Connection header received
0812 16:39:51.261778 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:51.261779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:51.261800 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:51.261880 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:39:51.262058 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.262531 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.263343 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.263526 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.263578 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:51.263736 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:39:51.263841 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:39:51.263903 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:51.263944 (+ 41us) server_negotiation.cc:299] Negotiation successful
0812 16:39:51.263963 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 16:39:51.266094 26800 raft_consensus.cc:1261] T 94b6c7a35ed247b29fb9cb779d051f1b P 2bcb06bb050f462bbe02533992ffb681 [term 1 FOLLOWER]: Refusing update from remote peer 1beb07e0229c436dba3ca8ab225dcbc2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:39:51.266481 26546 raft_consensus.cc:1261] T 94b6c7a35ed247b29fb9cb779d051f1b P 9503aa07ced9420fbf374cf0b694a872 [term 1 FOLLOWER]: Refusing update from remote peer 1beb07e0229c436dba3ca8ab225dcbc2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:39:51.266508 27196 consensus_queue.cc:1035] T 94b6c7a35ed247b29fb9cb779d051f1b P 1beb07e0229c436dba3ca8ab225dcbc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bcb06bb050f462bbe02533992ffb681" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:39:51.266822 27196 consensus_queue.cc:1035] T 94b6c7a35ed247b29fb9cb779d051f1b P 1beb07e0229c436dba3ca8ab225dcbc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9503aa07ced9420fbf374cf0b694a872" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 44207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:39:51.270057 27200 mvcc.cc:205] Tried to move back new op lower bound from 7317720233022533632 to 7317720232887328768. Current Snapshot: MvccSnapshot[applied={T|T < 7317720233022533632}]
I20260812 16:39:51.270216 27203 mvcc.cc:205] Tried to move back new op lower bound from 7317720233022533632 to 7317720232887328768. Current Snapshot: MvccSnapshot[applied={T|T < 7317720233022533632}]
I20260812 16:39:51.271363 27206 mvcc.cc:205] Tried to move back new op lower bound from 7317720233022533632 to 7317720232887328768. Current Snapshot: MvccSnapshot[applied={T|T < 7317720233022533632}]
I20260812 16:39:51.272756 26265 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51724:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:39:51.272863 26265 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51724:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:39:51.273593 26265 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe: Sending DeleteTablet for 3 replicas of tablet 94b6c7a35ed247b29fb9cb779d051f1b
I20260812 16:39:51.273938 26511 tablet_service.cc:1514] Processing DeleteTablet for tablet 94b6c7a35ed247b29fb9cb779d051f1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:39:51 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:50498
I20260812 16:39:51.273919 27028 tablet_service.cc:1514] Processing DeleteTablet for tablet 94b6c7a35ed247b29fb9cb779d051f1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:39:51 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:38644
I20260812 16:39:51.273979 26760 tablet_service.cc:1514] Processing DeleteTablet for tablet 94b6c7a35ed247b29fb9cb779d051f1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:39:51 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:58440
I20260812 16:39:51.274242 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 26385
I20260812 16:39:51.274260 27225 tablet_replica.cc:330] T 94b6c7a35ed247b29fb9cb779d051f1b P 1beb07e0229c436dba3ca8ab225dcbc2: stopping tablet replica
I20260812 16:39:51.291894 27228 tablet_replica.cc:330] T 94b6c7a35ed247b29fb9cb779d051f1b P 2bcb06bb050f462bbe02533992ffb681: stopping tablet replica
I20260812 16:39:51.292092 27228 raft_consensus.cc:2229] T 94b6c7a35ed247b29fb9cb779d051f1b P 2bcb06bb050f462bbe02533992ffb681 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:39:51.292234 27228 raft_consensus.cc:2258] T 94b6c7a35ed247b29fb9cb779d051f1b P 2bcb06bb050f462bbe02533992ffb681 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:39:51.292881 27228 ts_tablet_manager.cc:1905] T 94b6c7a35ed247b29fb9cb779d051f1b P 2bcb06bb050f462bbe02533992ffb681: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:39:51.295223 27228 ts_tablet_manager.cc:1918] T 94b6c7a35ed247b29fb9cb779d051f1b P 2bcb06bb050f462bbe02533992ffb681: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:39:51.295226 27225 raft_consensus.cc:2229] T 94b6c7a35ed247b29fb9cb779d051f1b P 1beb07e0229c436dba3ca8ab225dcbc2 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:39:51.295382 27225 raft_consensus.cc:2258] T 94b6c7a35ed247b29fb9cb779d051f1b P 1beb07e0229c436dba3ca8ab225dcbc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:39:51.295478 27228 log.cc:1196] T 94b6c7a35ed247b29fb9cb779d051f1b P 2bcb06bb050f462bbe02533992ffb681: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/wal/wals/94b6c7a35ed247b29fb9cb779d051f1b
I20260812 16:39:51.295849 27228 ts_tablet_manager.cc:1939] T 94b6c7a35ed247b29fb9cb779d051f1b P 2bcb06bb050f462bbe02533992ffb681: Deleting consensus metadata
I20260812 16:39:51.295887 27225 ts_tablet_manager.cc:1905] T 94b6c7a35ed247b29fb9cb779d051f1b P 1beb07e0229c436dba3ca8ab225dcbc2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:39:51.296492 26239 catalog_manager.cc:4841] TS 2bcb06bb050f462bbe02533992ffb681 (127.0.253.194:38999): tablet 94b6c7a35ed247b29fb9cb779d051f1b (table test-table [id=ca8aca19327f428993b54609993be76d]) successfully deleted
I20260812 16:39:51.297516 27225 ts_tablet_manager.cc:1918] T 94b6c7a35ed247b29fb9cb779d051f1b P 1beb07e0229c436dba3ca8ab225dcbc2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260812 16:39:51.297540 26241 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.253.193:44207 (error 108)
I20260812 16:39:51.297597 27225 log.cc:1196] T 94b6c7a35ed247b29fb9cb779d051f1b P 1beb07e0229c436dba3ca8ab225dcbc2: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/wal/wals/94b6c7a35ed247b29fb9cb779d051f1b
I20260812 16:39:51.297871 27225 ts_tablet_manager.cc:1939] T 94b6c7a35ed247b29fb9cb779d051f1b P 1beb07e0229c436dba3ca8ab225dcbc2: Deleting consensus metadata
I20260812 16:39:51.297891 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 26644
I20260812 16:39:51.301426 26243 catalog_manager.cc:4841] TS 1beb07e0229c436dba3ca8ab225dcbc2 (127.0.253.195:36289): tablet 94b6c7a35ed247b29fb9cb779d051f1b (table test-table [id=ca8aca19327f428993b54609993be76d]) successfully deleted
I20260812 16:39:51.302086 27168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:51.300986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:44207 (local address 127.0.0.1:50512)
0812 16:39:51.302010 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:51.302056 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.193:44207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1000}
W20260812 16:39:51.302292 26241 catalog_manager.cc:4584] TS 9503aa07ced9420fbf374cf0b694a872 (127.0.253.193:44207): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 94b6c7a35ed247b29fb9cb779d051f1b: Network error: Client connection negotiation failed: client connection to 127.0.253.193:44207: connect: Connection refused (error 111)
I20260812 16:39:51.310909 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 26900
I20260812 16:39:51.332461 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 26205
I20260812 16:39:51.351642 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:41633
--webserver_interface=127.0.253.254
--webserver_port=33581
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:38235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:41633
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:39:51.438463 27237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:39:51.438696 27237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:39:51.438720 27237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:39:51.439927 27237 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:39:51.439975 27237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:39:51.439989 27237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:39:51.440008 27237 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:39:51.440024 27237 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:39:51.441560 27237 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:38235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:41633
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:41633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=33581
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:39:51.441836 27237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:39:51.442092 27237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:39:51.451825 27250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:39:51.451826 27254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:39:51.451826 27249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:39:51.452111 27237 server_base.cc:1034] running on GCE node
I20260812 16:39:51.452414 27237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:39:51.452728 27237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:39:51.453886 27237 hybrid_clock.cc:648] HybridClock initialized: now 1786552791453870 us; error 46 us; skew 500 ppm
Aug 12 16:39:51 dist-test-slave-w4v5 krb5kdc[26187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552791, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:39:51.456647 27237 init.cc:377] Logged in from keytab as kudu/127.0.253.254@KRBTEST.COM (short username kudu)
I20260812 16:39:51.457204 27237 webserver.cc:460] Webserver started at http://127.0.253.254:33581/ using document root <none> and password file <none>
I20260812 16:39:51.457451 27237 fs_manager.cc:359] Metadata directory not provided
I20260812 16:39:51.457508 27237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:39:51.458964 27237 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:39:51.459733 27263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:51.459951 27237 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:39:51.460048 27237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "5c1bd1105c81469c9b3c1cf7dab17ffe"
format_stamp: "Formatted at 2026-08-12 16:39:50 on dist-test-slave-w4v5"
server_key: "b4af129b54d2289074366ee55d0dae69"
server_key_iv: "7822080e3266a7ba3984777f34d0abb6"
server_key_version: "encryptionkey@0"
I20260812 16:39:51.460516 27237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:39:51.478417 27237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:39:51.480542 27237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:39:51.480661 27237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:39:51.484771 27237 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:41633
I20260812 16:39:51.484820 27364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:41633 every 8 connection(s)
I20260812 16:39:51.485231 27237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:39:51.487015 27366 sys_catalog.cc:263] Verifying existing consensus state
I20260812 16:39:51.487507 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 27237
I20260812 16:39:51.487810 27366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe: Bootstrap starting.
I20260812 16:39:51.487847 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.193:44207
--local_ip_for_outbound_sockets=127.0.253.193
--tserver_master_addrs=127.0.253.254:41633
--webserver_port=37137
--webserver_interface=127.0.253.193
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:38235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:39:51.497076 27366 log.cc:824] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe: Log is configured to *not* fsync() on all Append() calls
I20260812 16:39:51.499142 27366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 16:39:51.499456 27366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe: Bootstrap complete.
I20260812 16:39:51.501781 27366 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1bd1105c81469c9b3c1cf7dab17ffe" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 41633 } }
I20260812 16:39:51.502036 27366 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c1bd1105c81469c9b3c1cf7dab17ffe, State: Initialized, Role: FOLLOWER
I20260812 16:39:51.502228 27366 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1bd1105c81469c9b3c1cf7dab17ffe" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 41633 } }
I20260812 16:39:51.502312 27366 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:39:51.502372 27366 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:39:51.502420 27366 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [term 1 FOLLOWER]: Advancing to term 2
I20260812 16:39:51.503587 27366 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1bd1105c81469c9b3c1cf7dab17ffe" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 41633 } }
I20260812 16:39:51.503719 27366 leader_election.cc:302] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c1bd1105c81469c9b3c1cf7dab17ffe; no voters:
I20260812 16:39:51.503929 27366 leader_election.cc:288] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 16:39:51.503968 27380 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [term 2 FOLLOWER]: Leader election won for term 2
I20260812 16:39:51.504082 27380 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [term 2 LEADER]: Becoming Leader. State: Replica: 5c1bd1105c81469c9b3c1cf7dab17ffe, State: Running, Role: LEADER
I20260812 16:39:51.504191 27380 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1bd1105c81469c9b3c1cf7dab17ffe" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 41633 } }
I20260812 16:39:51.504242 27366 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:39:51.504453 27380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c1bd1105c81469c9b3c1cf7dab17ffe. Latest consensus state: current_term: 2 leader_uuid: "5c1bd1105c81469c9b3c1cf7dab17ffe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1bd1105c81469c9b3c1cf7dab17ffe" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 41633 } } }
I20260812 16:39:51.504541 27380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [sys.catalog]: This master's current role is: LEADER
I20260812 16:39:51.504453 27382 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5c1bd1105c81469c9b3c1cf7dab17ffe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1bd1105c81469c9b3c1cf7dab17ffe" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 41633 } } }
I20260812 16:39:51.504711 27382 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [sys.catalog]: This master's current role is: LEADER
I20260812 16:39:51.504748 27387 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:39:51.505602 27387 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:39:51.505721 27387 catalog_manager.cc:1259] Loaded cluster ID: 4f3572aa723744668178e4247dc42536
I20260812 16:39:51.505764 27387 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:39:51.506162 27387 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260812 16:39:51.509014 27387 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260812 16:39:51.510870 27387 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:39:51.511109 27387 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe: Loaded TSK: 0
I20260812 16:39:51.511345 27387 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 16:39:51.589243 27373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:39:51.589496 27373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:39:51.589540 27373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:39:51.590695 27373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:39:51.590764 27373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:39:51.590811 27373 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.193
I20260812 16:39:51.592384 27373 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:38235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.193:44207
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.253.193
--webserver_port=37137
--tserver_master_addrs=127.0.253.254:41633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.193
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:39:51.592664 27373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:39:51.592900 27373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:39:51.598186 27423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:39:51.598310 27427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:39:51.598166 27422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:39:51.598415 27373 server_base.cc:1034] running on GCE node
I20260812 16:39:51.598627 27373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:39:51.598919 27373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:39:51.600121 27373 hybrid_clock.cc:648] HybridClock initialized: now 1786552791600106 us; error 29 us; skew 500 ppm
Aug 12 16:39:51 dist-test-slave-w4v5 krb5kdc[26187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552791, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:39:51.603374 27373 init.cc:377] Logged in from keytab as kudu/127.0.253.193@KRBTEST.COM (short username kudu)
I20260812 16:39:51.603933 27373 webserver.cc:460] Webserver started at http://127.0.253.193:37137/ using document root <none> and password file <none>
I20260812 16:39:51.604226 27373 fs_manager.cc:359] Metadata directory not provided
I20260812 16:39:51.604318 27373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:39:51.606189 27373 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 16:39:51.606992 27441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:51.607242 27373 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:39:51.607343 27373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/wal
uuid: "9503aa07ced9420fbf374cf0b694a872"
format_stamp: "Formatted at 2026-08-12 16:39:50 on dist-test-slave-w4v5"
server_key: "2d646c30b1b6ed0c6d8463ad168ff84b"
server_key_iv: "ecadb81110145fef90035ede117b1b14"
server_key_version: "encryptionkey@0"
I20260812 16:39:51.607800 27373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:39:51.629310 27373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:39:51.632079 27373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:39:51.632263 27373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:39:51.632565 27373 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:39:51.633131 27456 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 16:39:51.634879 27373 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 16:39:51.634946 27373 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 16:39:51.634984 27373 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 16:39:51.635948 27373 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 16:39:51.635998 27373 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 16:39:51.636133 27456 tablet_bootstrap.cc:492] T 94b6c7a35ed247b29fb9cb779d051f1b P 9503aa07ced9420fbf374cf0b694a872: Bootstrap starting.
I20260812 16:39:51.643213 27373 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.193:44207
I20260812 16:39:51.643440 27661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.193:44207 every 8 connection(s)
I20260812 16:39:51.643673 27373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
I20260812 16:39:51.644999 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 27373
I20260812 16:39:51.645335 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.194:38999
--local_ip_for_outbound_sockets=127.0.253.194
--tserver_master_addrs=127.0.253.254:41633
--webserver_port=42105
--webserver_interface=127.0.253.194
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:38235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:39:51.647864 27456 log.cc:824] T 94b6c7a35ed247b29fb9cb779d051f1b P 9503aa07ced9420fbf374cf0b694a872: Log is configured to *not* fsync() on all Append() calls
I20260812 16:39:51.650795 27456 tablet_bootstrap.cc:492] T 94b6c7a35ed247b29fb9cb779d051f1b P 9503aa07ced9420fbf374cf0b694a872: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 16:39:51.651178 27456 tablet_bootstrap.cc:492] T 94b6c7a35ed247b29fb9cb779d051f1b P 9503aa07ced9420fbf374cf0b694a872: Bootstrap complete.
I20260812 16:39:51.652442 27456 ts_tablet_manager.cc:1397] T 94b6c7a35ed247b29fb9cb779d051f1b P 9503aa07ced9420fbf374cf0b694a872: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.013s
I20260812 16:39:51.653541 27456 raft_consensus.cc:348] T 94b6c7a35ed247b29fb9cb779d051f1b P 9503aa07ced9420fbf374cf0b694a872 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beb07e0229c436dba3ca8ab225dcbc2" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 36289 } } peers { permanent_uuid: "9503aa07ced9420fbf374cf0b694a872" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 44207 } } peers { permanent_uuid: "2bcb06bb050f462bbe02533992ffb681" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38999 } }
I20260812 16:39:51.653877 27456 raft_consensus.cc:729] T 94b6c7a35ed247b29fb9cb779d051f1b P 9503aa07ced9420fbf374cf0b694a872 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9503aa07ced9420fbf374cf0b694a872, State: Initialized, Role: FOLLOWER
I20260812 16:39:51.654018 27456 consensus_queue.cc:260] T 94b6c7a35ed247b29fb9cb779d051f1b P 9503aa07ced9420fbf374cf0b694a872 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beb07e0229c436dba3ca8ab225dcbc2" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 36289 } } peers { permanent_uuid: "9503aa07ced9420fbf374cf0b694a872" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 44207 } } peers { permanent_uuid: "2bcb06bb050f462bbe02533992ffb681" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38999 } }
Aug 12 16:39:51 dist-test-slave-w4v5 krb5kdc[26187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552791, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:39:51.654317 27456 ts_tablet_manager.cc:1428] T 94b6c7a35ed247b29fb9cb779d051f1b P 9503aa07ced9420fbf374cf0b694a872: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260812 16:39:51.659214 27667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:51.644414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.193:42089)
0812 16:39:51.644606 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:51.644666 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 16:39:51.645784 (+ 1118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:39:51.646569 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:39:51.646573 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:39:51.647104 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:39:51.647427 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:51.647435 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.649546 (+ 2111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.649548 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:51.650952 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:51.650955 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.651950 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.651952 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:51.652093 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:51.652755 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:39:51.652765 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:39:51.654550 (+ 1785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:39:51.657121 (+ 2571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:51.657128 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:51.657130 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:51.657433 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:51.657730 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:51.657731 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:51.657732 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:51.657892 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:51.658369 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:39:51.658377 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:39:51.658558 (+ 181us) client_negotiation.cc:769] Sending connection context
0812 16:39:51.658699 (+ 141us) client_negotiation.cc:240] Negotiation successful
0812 16:39:51.658905 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":98,"threads_started":1}
I20260812 16:39:51.659317 27668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:51.644517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:42089 (local address 127.0.253.254:41633)
0812 16:39:51.646055 (+ 1538us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:51.646065 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:51.646112 (+ 47us) server_negotiation.cc:407] Connection header received
0812 16:39:51.646267 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:51.646278 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:51.646353 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:51.647572 (+ 1219us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:39:51.647680 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.649405 (+ 1725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.651050 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.651828 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.651935 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:51.654671 (+ 2736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:39:51.654676 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:39:51.654676 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:39:51.654689 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:39:51.657028 (+ 2339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:51.657535 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:51.657538 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:51.657539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:51.657653 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:51.657982 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:51.657983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:51.657984 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:51.658280 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:39:51.658370 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:51.658775 (+ 405us) server_negotiation.cc:299] Negotiation successful
0812 16:39:51.658920 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1481,"thread_start_us":92,"threads_started":1}
I20260812 16:39:51.660561 27662 heartbeater.cc:347] Connected to a master server at 127.0.253.254:41633
I20260812 16:39:51.660673 27662 heartbeater.cc:464] Registering TS with master...
I20260812 16:39:51.660880 27662 heartbeater.cc:511] Master 127.0.253.254:41633 requested a full tablet report, sending...
I20260812 16:39:51.661795 27300 ts_manager.cc:194] Registered new tserver with Master: 9503aa07ced9420fbf374cf0b694a872 (127.0.253.193:44207)
I20260812 16:39:51.663126 27300 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.193@KRBTEST.COM'} at 127.0.253.193:42089
I20260812 16:39:51.664299 27662 heartbeater.cc:503] Master 127.0.253.254:41633 was elected leader, sending a full tablet report...
I20260812 16:39:51.671221 27672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:51.662508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:44207 (local address 127.0.0.1:50522)
0812 16:39:51.667171 (+ 4663us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:51.667201 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0812 16:39:51.667294 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:39:51.667539 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:39:51.667543 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:39:51.667545 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:39:51.667615 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:51.667616 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.668699 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.668700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:51.670003 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:51.670006 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.670882 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.670883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:51.670972 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:51.670985 (+ 13us) client_negotiation.cc:769] Sending connection context
0812 16:39:51.671063 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 16:39:51.671139 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4628,"thread_start_us":97,"threads_started":1}
I20260812 16:39:51.671248 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:51.662773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50522 (local address 127.0.253.193:44207)
0812 16:39:51.662932 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:51.662937 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:51.667275 (+ 4338us) server_negotiation.cc:407] Connection header received
0812 16:39:51.667399 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:51.667402 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:51.667422 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:51.667510 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:39:51.667711 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.668613 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.670103 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.670786 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.670847 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:51.670884 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:51.671138 (+ 254us) server_negotiation.cc:299] Negotiation successful
0812 16:39:51.671179 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":72,"threads_started":1}
I20260812 16:39:51.671744 27538 tablet_service.cc:1514] Processing DeleteTablet for tablet 94b6c7a35ed247b29fb9cb779d051f1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:39:51 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:50522
I20260812 16:39:51.672322 27675 tablet_replica.cc:330] stopping tablet replica
I20260812 16:39:51.672444 27675 raft_consensus.cc:2229] T 94b6c7a35ed247b29fb9cb779d051f1b P 9503aa07ced9420fbf374cf0b694a872 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:39:51.672536 27675 raft_consensus.cc:2258] T 94b6c7a35ed247b29fb9cb779d051f1b P 9503aa07ced9420fbf374cf0b694a872 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:39:51.672870 27675 ts_tablet_manager.cc:1905] T 94b6c7a35ed247b29fb9cb779d051f1b P 9503aa07ced9420fbf374cf0b694a872: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:39:51.674829 27675 ts_tablet_manager.cc:1918] T 94b6c7a35ed247b29fb9cb779d051f1b P 9503aa07ced9420fbf374cf0b694a872: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:39:51.674925 27675 log.cc:1196] T 94b6c7a35ed247b29fb9cb779d051f1b P 9503aa07ced9420fbf374cf0b694a872: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/wal/wals/94b6c7a35ed247b29fb9cb779d051f1b
I20260812 16:39:51.675231 27675 ts_tablet_manager.cc:1939] T 94b6c7a35ed247b29fb9cb779d051f1b P 9503aa07ced9420fbf374cf0b694a872: Deleting consensus metadata
I20260812 16:39:51.675746 27274 catalog_manager.cc:4841] TS 9503aa07ced9420fbf374cf0b694a872 (127.0.253.193:44207): tablet 94b6c7a35ed247b29fb9cb779d051f1b (table test-table [id=ca8aca19327f428993b54609993be76d]) successfully deleted
W20260812 16:39:51.741266 27669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:39:51.741533 27669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:39:51.741573 27669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:39:51.742738 27669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:39:51.742812 27669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:39:51.742856 27669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.194
I20260812 16:39:51.744378 27669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:38235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.194:38999
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.253.194
--webserver_port=42105
--tserver_master_addrs=127.0.253.254:41633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.194
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:39:51.744655 27669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:39:51.744896 27669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:39:51.750319 27687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:39:51.750329 27694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:39:51.750363 27669 server_base.cc:1034] running on GCE node
W20260812 16:39:51.750329 27690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:39:51.750844 27669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:39:51.751045 27669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:39:51.752241 27669 hybrid_clock.cc:648] HybridClock initialized: now 1786552791752217 us; error 42 us; skew 500 ppm
Aug 12 16:39:51 dist-test-slave-w4v5 krb5kdc[26187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552791, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:39:51.754900 27669 init.cc:377] Logged in from keytab as kudu/127.0.253.194@KRBTEST.COM (short username kudu)
I20260812 16:39:51.755384 27669 webserver.cc:460] Webserver started at http://127.0.253.194:42105/ using document root <none> and password file <none>
I20260812 16:39:51.755592 27669 fs_manager.cc:359] Metadata directory not provided
I20260812 16:39:51.755649 27669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:39:51.757105 27669 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:39:51.757890 27712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:51.758190 27669 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:39:51.758282 27669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/wal
uuid: "2bcb06bb050f462bbe02533992ffb681"
format_stamp: "Formatted at 2026-08-12 16:39:50 on dist-test-slave-w4v5"
server_key: "f334eca05f1bda10a835ce781e2d5e46"
server_key_iv: "a369b104688bd16f17ffd88c8149316d"
server_key_version: "encryptionkey@0"
I20260812 16:39:51.758600 27669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:39:51.780895 27669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:39:51.782897 27669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:39:51.783010 27669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:39:51.783289 27669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:39:51.783632 27669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:39:51.783690 27669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:51.783726 27669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:39:51.783771 27669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:51.789664 27669 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.194:38999
I20260812 16:39:51.789738 27934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.194:38999 every 8 connection(s)
I20260812 16:39:51.790102 27669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
Aug 12 16:39:51 dist-test-slave-w4v5 krb5kdc[26187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552791, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:39:51.798879 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 27669
I20260812 16:39:51.799257 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.195:36289
--local_ip_for_outbound_sockets=127.0.253.195
--tserver_master_addrs=127.0.253.254:41633
--webserver_port=45895
--webserver_interface=127.0.253.195
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:38235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:39:51.799928 27668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:51.790932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:53507 (local address 127.0.253.254:41633)
0812 16:39:51.791015 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:51.791017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:51.791256 (+ 239us) server_negotiation.cc:407] Connection header received
0812 16:39:51.791802 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:51.791803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:51.791819 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:51.791872 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:39:51.792894 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.793406 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.794453 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.794683 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.794774 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:51.797038 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:39:51.797040 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:39:51.797040 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:39:51.797047 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:39:51.798271 (+ 1224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:51.798762 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:51.798764 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:51.798765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:51.798836 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:51.799178 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:51.799179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:51.799179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:51.799339 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:39:51.799417 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:51.799779 (+ 362us) server_negotiation.cc:299] Negotiation successful
0812 16:39:51.799862 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 16:39:51.800230 27938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:51.790914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.194:53507)
0812 16:39:51.791123 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:51.791169 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 16:39:51.791685 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:39:51.791956 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:39:51.791959 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:39:51.792580 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:39:51.792778 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:51.792783 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.793512 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.793513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:51.794344 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:51.794347 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.794788 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.794789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:51.794913 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:51.795495 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:39:51.795504 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:39:51.796946 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:39:51.798364 (+ 1418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:51.798372 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:51.798374 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:51.798656 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:51.798920 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:51.798921 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:51.798921 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:51.799089 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:51.799451 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:39:51.799458 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:39:51.799622 (+ 164us) client_negotiation.cc:769] Sending connection context
0812 16:39:51.799726 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 16:39:51.799912 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":65,"threads_started":1}
I20260812 16:39:51.800817 27936 heartbeater.cc:347] Connected to a master server at 127.0.253.254:41633
I20260812 16:39:51.800915 27936 heartbeater.cc:464] Registering TS with master...
I20260812 16:39:51.801143 27936 heartbeater.cc:511] Master 127.0.253.254:41633 requested a full tablet report, sending...
I20260812 16:39:51.801640 27300 ts_manager.cc:194] Registered new tserver with Master: 2bcb06bb050f462bbe02533992ffb681 (127.0.253.194:38999)
I20260812 16:39:51.802058 27300 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.194@KRBTEST.COM'} at 127.0.253.194:53507
W20260812 16:39:51.884265 27940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:39:51.884466 27940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:39:51.884490 27940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:39:51.885634 27940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:39:51.885677 27940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:39:51.885730 27940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.195
I20260812 16:39:51.887300 27940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:38235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.195:36289
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.253.195
--webserver_port=45895
--tserver_master_addrs=127.0.253.254:41633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.195
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:39:51.887557 27940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:39:51.887797 27940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:39:51.892975 27950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:39:51.892975 27959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:39:51.893023 27955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:39:51.893206 27940 server_base.cc:1034] running on GCE node
I20260812 16:39:51.893407 27940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:39:51.893651 27940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:39:51.894811 27940 hybrid_clock.cc:648] HybridClock initialized: now 1786552791894790 us; error 33 us; skew 500 ppm
Aug 12 16:39:51 dist-test-slave-w4v5 krb5kdc[26187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552791, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:39:51.897517 27940 init.cc:377] Logged in from keytab as kudu/127.0.253.195@KRBTEST.COM (short username kudu)
I20260812 16:39:51.898010 27940 webserver.cc:460] Webserver started at http://127.0.253.195:45895/ using document root <none> and password file <none>
I20260812 16:39:51.898325 27940 fs_manager.cc:359] Metadata directory not provided
I20260812 16:39:51.898430 27940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:39:51.899902 27940 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:39:51.900606 27982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:51.900803 27940 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:39:51.900914 27940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/wal
uuid: "1beb07e0229c436dba3ca8ab225dcbc2"
format_stamp: "Formatted at 2026-08-12 16:39:50 on dist-test-slave-w4v5"
server_key: "2eb8de141d7e4d20cd6f3732316a1be2"
server_key_iv: "cc52484f739097cf29b3d5e1a3d4517b"
server_key_version: "encryptionkey@0"
I20260812 16:39:51.901271 27940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:39:51.926242 27940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:39:51.928763 27940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:39:51.928937 27940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:39:51.929225 27940 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:39:51.929557 27940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:39:51.929611 27940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:51.929649 27940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:39:51.929692 27940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:51.935531 27940 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.195:36289
I20260812 16:39:51.935637 28216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.195:36289 every 8 connection(s)
I20260812 16:39:51.936033 27940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
Aug 12 16:39:51 dist-test-slave-w4v5 krb5kdc[26187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552791, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:39:51.945247 27668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:51.936828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:55607 (local address 127.0.253.254:41633)
0812 16:39:51.936897 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:51.936899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:51.937096 (+ 197us) server_negotiation.cc:407] Connection header received
0812 16:39:51.937596 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:51.937597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:51.937608 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:51.937684 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:39:51.938724 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.939281 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.940234 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.940418 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.940472 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:51.942593 (+ 2121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:39:51.942594 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:39:51.942594 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:39:51.942601 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:39:51.943971 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:51.944360 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:51.944361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:51.944361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:51.944411 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:51.944665 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:51.944666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:51.944666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:51.944768 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:39:51.944818 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:51.945123 (+ 305us) server_negotiation.cc:299] Negotiation successful
0812 16:39:51.945184 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 16:39:51.945492 28221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:51.936836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.195:55607)
0812 16:39:51.936989 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:51.937025 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 16:39:51.937503 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:39:51.937742 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:39:51.937745 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:39:51.938424 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:39:51.938618 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:51.938623 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.939360 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.939360 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:51.940157 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:51.940159 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.940491 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.940492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:51.940597 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:51.941128 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:39:51.941150 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:39:51.942509 (+ 1359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:39:51.944058 (+ 1549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:51.944065 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:51.944067 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:51.944290 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:51.944472 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:51.944472 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:51.944473 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:51.944589 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:51.944857 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:39:51.944863 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:39:51.944991 (+ 128us) client_negotiation.cc:769] Sending connection context
0812 16:39:51.945098 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 16:39:51.945221 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":74,"threads_started":1}
I20260812 16:39:51.946053 28217 heartbeater.cc:347] Connected to a master server at 127.0.253.254:41633
I20260812 16:39:51.946179 28217 heartbeater.cc:464] Registering TS with master...
I20260812 16:39:51.946206 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 27940
I20260812 16:39:51.946408 28217 heartbeater.cc:511] Master 127.0.253.254:41633 requested a full tablet report, sending...
I20260812 16:39:51.946841 27300 ts_manager.cc:194] Registered new tserver with Master: 1beb07e0229c436dba3ca8ab225dcbc2 (127.0.253.195:36289)
I20260812 16:39:51.947376 27300 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.195@KRBTEST.COM'} at 127.0.253.195:55607
I20260812 16:39:51.952670 27668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:51.946731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51728 (local address 127.0.253.254:41633)
0812 16:39:51.946778 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:51.946781 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:51.947002 (+ 221us) server_negotiation.cc:407] Connection header received
0812 16:39:51.947025 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:51.947026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:51.947034 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:51.947144 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:39:51.947887 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.948336 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.949071 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.949244 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.949299 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:51.950366 (+ 1067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:39:51.950367 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:39:51.950367 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:39:51.950372 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:39:51.951779 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:51.952074 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:51.952075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:51.952075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:51.952123 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:51.952302 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:51.952303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:51.952303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:51.952393 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:39:51.952510 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:51.952561 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 16:39:51.952615 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 16:39:51.953476 1015 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:39:51.956287 27668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:51.953775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51734 (local address 127.0.253.254:41633)
0812 16:39:51.953810 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:51.953812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:51.953965 (+ 153us) server_negotiation.cc:407] Connection header received
0812 16:39:51.954038 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:51.954039 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:51.954070 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:51.954171 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:39:51.954269 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.954711 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.955531 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.955734 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.955820 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:51.955988 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:39:51.956087 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:39:51.956181 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:51.956201 (+ 20us) server_negotiation.cc:299] Negotiation successful
0812 16:39:51.956218 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 16:39:51.956605 27300 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:51734:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:39:51.958201 27300 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:39:51.960539 27538 tablet_service.cc:1467] Processing CreateTablet for tablet b799abf0b5b14cfb8070b576480c3eb5 (DEFAULT_TABLE table=test-table [id=71cc5548f2314bdeb7373c133c8b8db4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:39:51.960748 27538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b799abf0b5b14cfb8070b576480c3eb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:39:51.962234 27456 tablet_bootstrap.cc:492] T b799abf0b5b14cfb8070b576480c3eb5 P 9503aa07ced9420fbf374cf0b694a872: Bootstrap starting.
I20260812 16:39:51.962646 27456 tablet_bootstrap.cc:654] T b799abf0b5b14cfb8070b576480c3eb5 P 9503aa07ced9420fbf374cf0b694a872: Neither blocks nor log segments found. Creating new log.
I20260812 16:39:51.963193 27456 tablet_bootstrap.cc:492] T b799abf0b5b14cfb8070b576480c3eb5 P 9503aa07ced9420fbf374cf0b694a872: No bootstrap required, opened a new log
I20260812 16:39:51.963245 27456 ts_tablet_manager.cc:1397] T b799abf0b5b14cfb8070b576480c3eb5 P 9503aa07ced9420fbf374cf0b694a872: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 16:39:51.963356 27456 raft_consensus.cc:348] T b799abf0b5b14cfb8070b576480c3eb5 P 9503aa07ced9420fbf374cf0b694a872 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beb07e0229c436dba3ca8ab225dcbc2" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 36289 } } peers { permanent_uuid: "2bcb06bb050f462bbe02533992ffb681" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38999 } } peers { permanent_uuid: "9503aa07ced9420fbf374cf0b694a872" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 44207 } }
I20260812 16:39:51.963444 27456 raft_consensus.cc:374] T b799abf0b5b14cfb8070b576480c3eb5 P 9503aa07ced9420fbf374cf0b694a872 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:39:51.963491 27456 raft_consensus.cc:729] T b799abf0b5b14cfb8070b576480c3eb5 P 9503aa07ced9420fbf374cf0b694a872 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9503aa07ced9420fbf374cf0b694a872, State: Initialized, Role: FOLLOWER
I20260812 16:39:51.963543 27456 consensus_queue.cc:260] T b799abf0b5b14cfb8070b576480c3eb5 P 9503aa07ced9420fbf374cf0b694a872 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beb07e0229c436dba3ca8ab225dcbc2" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 36289 } } peers { permanent_uuid: "2bcb06bb050f462bbe02533992ffb681" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38999 } } peers { permanent_uuid: "9503aa07ced9420fbf374cf0b694a872" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 44207 } }
I20260812 16:39:51.963649 27456 ts_tablet_manager.cc:1428] T b799abf0b5b14cfb8070b576480c3eb5 P 9503aa07ced9420fbf374cf0b694a872: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 16:39:51.964679 28230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:51.960380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38658 (local address 127.0.253.195:36289)
0812 16:39:51.960618 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:51.960623 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:51.960879 (+ 256us) server_negotiation.cc:407] Connection header received
0812 16:39:51.960918 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:51.960920 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:51.960936 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:51.960999 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:39:51.961199 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.962291 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.963612 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.964313 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.964392 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:51.964456 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:51.964560 (+ 104us) server_negotiation.cc:299] Negotiation successful
0812 16:39:51.964601 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":101,"threads_started":1}
I20260812 16:39:51.964670 28234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:51.960276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.195:36289 (local address 127.0.0.1:38658)
0812 16:39:51.960755 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:51.960765 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:39:51.960819 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:39:51.961068 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:39:51.961069 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:39:51.961071 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:39:51.961113 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:51.961114 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.962386 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.962387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:51.963526 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:51.963528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.964390 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.964391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:51.964472 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:51.964477 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:39:51.964545 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 16:39:51.964589 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"mutex_wait_us":380,"thread_start_us":47,"threads_started":1}
I20260812 16:39:51.964775 28232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:51.960510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58446 (local address 127.0.253.194:38999)
0812 16:39:51.960729 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:51.960735 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:51.960747 (+ 12us) server_negotiation.cc:407] Connection header received
0812 16:39:51.960792 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:51.960794 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:51.960811 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:51.960924 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:39:51.961047 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.962165 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.963636 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.964382 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.964445 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:51.964493 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:51.964639 (+ 146us) server_negotiation.cc:299] Negotiation successful
0812 16:39:51.964682 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":77,"threads_started":1}
I20260812 16:39:51.964789 27672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:51.960269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:38999 (local address 127.0.0.1:58446)
0812 16:39:51.960328 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:51.960338 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:39:51.960377 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:39:51.960931 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:39:51.960933 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:39:51.960935 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:39:51.960976 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:51.960980 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.962276 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.962277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:51.963437 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:51.963440 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.964452 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.964452 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:51.964504 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:51.964506 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 16:39:51.964628 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 16:39:51.964662 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 16:39:51.965885 28069 tablet_service.cc:1467] Processing CreateTablet for tablet b799abf0b5b14cfb8070b576480c3eb5 (DEFAULT_TABLE table=test-table [id=71cc5548f2314bdeb7373c133c8b8db4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:39:51.965883 27802 tablet_service.cc:1467] Processing CreateTablet for tablet b799abf0b5b14cfb8070b576480c3eb5 (DEFAULT_TABLE table=test-table [id=71cc5548f2314bdeb7373c133c8b8db4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:39:51.966210 28069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b799abf0b5b14cfb8070b576480c3eb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:39:51.966213 27802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b799abf0b5b14cfb8070b576480c3eb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:39:51.969246 28238 tablet_bootstrap.cc:492] T b799abf0b5b14cfb8070b576480c3eb5 P 1beb07e0229c436dba3ca8ab225dcbc2: Bootstrap starting.
I20260812 16:39:51.969769 28238 tablet_bootstrap.cc:654] T b799abf0b5b14cfb8070b576480c3eb5 P 1beb07e0229c436dba3ca8ab225dcbc2: Neither blocks nor log segments found. Creating new log.
I20260812 16:39:51.970333 28238 log.cc:824] T b799abf0b5b14cfb8070b576480c3eb5 P 1beb07e0229c436dba3ca8ab225dcbc2: Log is configured to *not* fsync() on all Append() calls
I20260812 16:39:51.970613 28236 tablet_bootstrap.cc:492] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681: Bootstrap starting.
I20260812 16:39:51.971201 28238 tablet_bootstrap.cc:492] T b799abf0b5b14cfb8070b576480c3eb5 P 1beb07e0229c436dba3ca8ab225dcbc2: No bootstrap required, opened a new log
I20260812 16:39:51.971223 28236 tablet_bootstrap.cc:654] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681: Neither blocks nor log segments found. Creating new log.
I20260812 16:39:51.971259 28238 ts_tablet_manager.cc:1397] T b799abf0b5b14cfb8070b576480c3eb5 P 1beb07e0229c436dba3ca8ab225dcbc2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:39:51.971709 28236 log.cc:824] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681: Log is configured to *not* fsync() on all Append() calls
I20260812 16:39:51.972396 28236 tablet_bootstrap.cc:492] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681: No bootstrap required, opened a new log
I20260812 16:39:51.972477 28236 ts_tablet_manager.cc:1397] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260812 16:39:51.972682 28238 raft_consensus.cc:348] T b799abf0b5b14cfb8070b576480c3eb5 P 1beb07e0229c436dba3ca8ab225dcbc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beb07e0229c436dba3ca8ab225dcbc2" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 36289 } } peers { permanent_uuid: "2bcb06bb050f462bbe02533992ffb681" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38999 } } peers { permanent_uuid: "9503aa07ced9420fbf374cf0b694a872" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 44207 } }
I20260812 16:39:51.972819 28238 raft_consensus.cc:374] T b799abf0b5b14cfb8070b576480c3eb5 P 1beb07e0229c436dba3ca8ab225dcbc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:39:51.972841 28238 raft_consensus.cc:729] T b799abf0b5b14cfb8070b576480c3eb5 P 1beb07e0229c436dba3ca8ab225dcbc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1beb07e0229c436dba3ca8ab225dcbc2, State: Initialized, Role: FOLLOWER
I20260812 16:39:51.972980 28238 consensus_queue.cc:260] T b799abf0b5b14cfb8070b576480c3eb5 P 1beb07e0229c436dba3ca8ab225dcbc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beb07e0229c436dba3ca8ab225dcbc2" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 36289 } } peers { permanent_uuid: "2bcb06bb050f462bbe02533992ffb681" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38999 } } peers { permanent_uuid: "9503aa07ced9420fbf374cf0b694a872" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 44207 } }
I20260812 16:39:51.973286 28238 ts_tablet_manager.cc:1428] T b799abf0b5b14cfb8070b576480c3eb5 P 1beb07e0229c436dba3ca8ab225dcbc2: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 16:39:51.973294 28217 heartbeater.cc:503] Master 127.0.253.254:41633 was elected leader, sending a full tablet report...
I20260812 16:39:51.974179 28236 raft_consensus.cc:348] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beb07e0229c436dba3ca8ab225dcbc2" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 36289 } } peers { permanent_uuid: "2bcb06bb050f462bbe02533992ffb681" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38999 } } peers { permanent_uuid: "9503aa07ced9420fbf374cf0b694a872" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 44207 } }
I20260812 16:39:51.974329 28236 raft_consensus.cc:374] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:39:51.974375 28236 raft_consensus.cc:729] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bcb06bb050f462bbe02533992ffb681, State: Initialized, Role: FOLLOWER
I20260812 16:39:51.974486 28236 consensus_queue.cc:260] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beb07e0229c436dba3ca8ab225dcbc2" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 36289 } } peers { permanent_uuid: "2bcb06bb050f462bbe02533992ffb681" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38999 } } peers { permanent_uuid: "9503aa07ced9420fbf374cf0b694a872" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 44207 } }
I20260812 16:39:51.974738 28236 ts_tablet_manager.cc:1428] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260812 16:39:51.974803 27936 heartbeater.cc:503] Master 127.0.253.254:41633 was elected leader, sending a full tablet report...
I20260812 16:39:51.976783 28242 raft_consensus.cc:482] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:39:51.976850 28242 raft_consensus.cc:504] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beb07e0229c436dba3ca8ab225dcbc2" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 36289 } } peers { permanent_uuid: "2bcb06bb050f462bbe02533992ffb681" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38999 } } peers { permanent_uuid: "9503aa07ced9420fbf374cf0b694a872" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 44207 } }
I20260812 16:39:51.977094 28242 leader_election.cc:288] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1beb07e0229c436dba3ca8ab225dcbc2 (127.0.253.195:36289), 9503aa07ced9420fbf374cf0b694a872 (127.0.253.193:44207)
I20260812 16:39:51.980055 28230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:51.977295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:35393 (local address 127.0.253.195:36289)
0812 16:39:51.977331 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:51.977333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:51.977345 (+ 12us) server_negotiation.cc:407] Connection header received
0812 16:39:51.977441 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:51.977443 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:51.977453 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:51.977501 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:39:51.977695 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.978308 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.979306 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.979715 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.979770 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:51.979777 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:51.979953 (+ 176us) server_negotiation.cc:299] Negotiation successful
0812 16:39:51.979995 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 16:39:51.980053 27938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:51.977247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.195:36289 (local address 127.0.253.194:35393)
0812 16:39:51.977305 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:51.977316 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:39:51.977355 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:39:51.977542 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:39:51.977543 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:39:51.977545 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:39:51.977617 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:51.977618 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.978386 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.978387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:51.979228 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:51.979230 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.979791 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.979792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:51.979878 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:51.979882 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:39:51.979931 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 16:39:51.979965 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":1}
I20260812 16:39:51.980150 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:51.977353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:51785 (local address 127.0.253.193:44207)
0812 16:39:51.977391 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:51.977393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:51.977450 (+ 57us) server_negotiation.cc:407] Connection header received
0812 16:39:51.977542 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:51.977543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:51.977553 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:51.977617 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:39:51.977779 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.978406 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.979402 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.979782 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.979852 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:51.979861 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:51.980031 (+ 170us) server_negotiation.cc:299] Negotiation successful
0812 16:39:51.980079 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 16:39:51.980173 28246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:51.977247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:44207 (local address 127.0.253.194:51785)
0812 16:39:51.977402 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:51.977412 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:39:51.977462 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:39:51.977625 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:39:51.977625 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:39:51.977626 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:39:51.977706 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:51.977708 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.978489 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.978489 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:51.979319 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:51.979321 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.979871 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.979872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:51.979951 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:51.979956 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:39:51.980011 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 16:39:51.980046 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":24,"thread_start_us":82,"threads_started":1}
I20260812 16:39:51.980414 27568 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b799abf0b5b14cfb8070b576480c3eb5" candidate_uuid: "2bcb06bb050f462bbe02533992ffb681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9503aa07ced9420fbf374cf0b694a872" is_pre_election: true
I20260812 16:39:51.980505 28110 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b799abf0b5b14cfb8070b576480c3eb5" candidate_uuid: "2bcb06bb050f462bbe02533992ffb681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1beb07e0229c436dba3ca8ab225dcbc2" is_pre_election: true
I20260812 16:39:51.980576 27568 raft_consensus.cc:2454] T b799abf0b5b14cfb8070b576480c3eb5 P 9503aa07ced9420fbf374cf0b694a872 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2bcb06bb050f462bbe02533992ffb681 in term 0.
I20260812 16:39:51.980605 28110 raft_consensus.cc:2454] T b799abf0b5b14cfb8070b576480c3eb5 P 1beb07e0229c436dba3ca8ab225dcbc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2bcb06bb050f462bbe02533992ffb681 in term 0.
I20260812 16:39:51.980831 27718 leader_election.cc:302] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1beb07e0229c436dba3ca8ab225dcbc2, 2bcb06bb050f462bbe02533992ffb681; no voters:
I20260812 16:39:51.980955 28242 raft_consensus.cc:2781] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:39:51.981009 28242 raft_consensus.cc:482] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:39:51.981030 28242 raft_consensus.cc:3037] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:39:51.981607 28242 raft_consensus.cc:504] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beb07e0229c436dba3ca8ab225dcbc2" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 36289 } } peers { permanent_uuid: "2bcb06bb050f462bbe02533992ffb681" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38999 } } peers { permanent_uuid: "9503aa07ced9420fbf374cf0b694a872" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 44207 } }
I20260812 16:39:51.981743 28242 leader_election.cc:288] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681 [CANDIDATE]: Term 1 election: Requested vote from peers 1beb07e0229c436dba3ca8ab225dcbc2 (127.0.253.195:36289), 9503aa07ced9420fbf374cf0b694a872 (127.0.253.193:44207)
I20260812 16:39:51.981884 28110 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b799abf0b5b14cfb8070b576480c3eb5" candidate_uuid: "2bcb06bb050f462bbe02533992ffb681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1beb07e0229c436dba3ca8ab225dcbc2"
I20260812 16:39:51.981889 27568 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b799abf0b5b14cfb8070b576480c3eb5" candidate_uuid: "2bcb06bb050f462bbe02533992ffb681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9503aa07ced9420fbf374cf0b694a872"
I20260812 16:39:51.981957 28110 raft_consensus.cc:3037] T b799abf0b5b14cfb8070b576480c3eb5 P 1beb07e0229c436dba3ca8ab225dcbc2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:39:51.981958 27568 raft_consensus.cc:3037] T b799abf0b5b14cfb8070b576480c3eb5 P 9503aa07ced9420fbf374cf0b694a872 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:39:51.982844 28110 raft_consensus.cc:2454] T b799abf0b5b14cfb8070b576480c3eb5 P 1beb07e0229c436dba3ca8ab225dcbc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2bcb06bb050f462bbe02533992ffb681 in term 1.
I20260812 16:39:51.983098 27718 leader_election.cc:302] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1beb07e0229c436dba3ca8ab225dcbc2, 2bcb06bb050f462bbe02533992ffb681; no voters:
I20260812 16:39:51.983246 27568 raft_consensus.cc:2454] T b799abf0b5b14cfb8070b576480c3eb5 P 9503aa07ced9420fbf374cf0b694a872 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2bcb06bb050f462bbe02533992ffb681 in term 1.
I20260812 16:39:51.983254 28242 raft_consensus.cc:2781] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:39:51.983443 28242 raft_consensus.cc:686] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681 [term 1 LEADER]: Becoming Leader. State: Replica: 2bcb06bb050f462bbe02533992ffb681, State: Running, Role: LEADER
I20260812 16:39:51.983536 28242 consensus_queue.cc:237] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beb07e0229c436dba3ca8ab225dcbc2" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 36289 } } peers { permanent_uuid: "2bcb06bb050f462bbe02533992ffb681" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38999 } } peers { permanent_uuid: "9503aa07ced9420fbf374cf0b694a872" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 44207 } }
I20260812 16:39:51.984301 27300 catalog_manager.cc:5495] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2bcb06bb050f462bbe02533992ffb681 (127.0.253.194). New cstate: current_term: 1 leader_uuid: "2bcb06bb050f462bbe02533992ffb681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beb07e0229c436dba3ca8ab225dcbc2" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 36289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2bcb06bb050f462bbe02533992ffb681" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9503aa07ced9420fbf374cf0b694a872" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 44207 } health_report { overall_health: UNKNOWN } } }
I20260812 16:39:51.993901 28232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:51.991430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58462 (local address 127.0.253.194:38999)
0812 16:39:51.991479 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:51.991481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:51.991488 (+ 7us) server_negotiation.cc:407] Connection header received
0812 16:39:51.991553 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:51.991554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:51.991569 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:51.991626 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:39:51.991723 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.992160 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.992867 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:51.993042 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:51.993092 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:51.993458 (+ 366us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:39:51.993553 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:39:51.993620 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:51.993759 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 16:39:51.993777 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 16:39:51.995820 28110 raft_consensus.cc:1261] T b799abf0b5b14cfb8070b576480c3eb5 P 1beb07e0229c436dba3ca8ab225dcbc2 [term 1 FOLLOWER]: Refusing update from remote peer 2bcb06bb050f462bbe02533992ffb681: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:39:51.995829 27568 raft_consensus.cc:1261] T b799abf0b5b14cfb8070b576480c3eb5 P 9503aa07ced9420fbf374cf0b694a872 [term 1 FOLLOWER]: Refusing update from remote peer 2bcb06bb050f462bbe02533992ffb681: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:39:51.996213 28248 consensus_queue.cc:1035] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9503aa07ced9420fbf374cf0b694a872" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 44207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:39:51.996351 28242 consensus_queue.cc:1035] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1beb07e0229c436dba3ca8ab225dcbc2" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 36289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:39:51.999229 28252 mvcc.cc:205] Tried to move back new op lower bound from 7317720236012158976 to 7317720235965112320. Current Snapshot: MvccSnapshot[applied={T|T < 7317720236012158976 or (T in {7317720236012158976})}]
I20260812 16:39:51.999703 28253 mvcc.cc:205] Tried to move back new op lower bound from 7317720236012158976 to 7317720235965112320. Current Snapshot: MvccSnapshot[applied={T|T < 7317720236012158976 or (T in {7317720236012158976})}]
I20260812 16:39:52.000448 27300 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:51734:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:39:52.000547 27300 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:51734:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:39:52.001262 27300 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe: Sending DeleteTablet for 3 replicas of tablet b799abf0b5b14cfb8070b576480c3eb5
I20260812 16:39:52.001562 27802 tablet_service.cc:1514] Processing DeleteTablet for tablet b799abf0b5b14cfb8070b576480c3eb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:39:51 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:58446
I20260812 16:39:52.001586 28069 tablet_service.cc:1514] Processing DeleteTablet for tablet b799abf0b5b14cfb8070b576480c3eb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:39:51 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:38658
I20260812 16:39:52.001798 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 27373
I20260812 16:39:52.001897 28275 tablet_replica.cc:330] T b799abf0b5b14cfb8070b576480c3eb5 P 1beb07e0229c436dba3ca8ab225dcbc2: stopping tablet replica
I20260812 16:39:52.001984 28275 raft_consensus.cc:2229] T b799abf0b5b14cfb8070b576480c3eb5 P 1beb07e0229c436dba3ca8ab225dcbc2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:39:52.002071 28275 raft_consensus.cc:2258] T b799abf0b5b14cfb8070b576480c3eb5 P 1beb07e0229c436dba3ca8ab225dcbc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:39:52.020493 28275 ts_tablet_manager.cc:1905] T b799abf0b5b14cfb8070b576480c3eb5 P 1beb07e0229c436dba3ca8ab225dcbc2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:39:52.022250 28275 ts_tablet_manager.cc:1918] T b799abf0b5b14cfb8070b576480c3eb5 P 1beb07e0229c436dba3ca8ab225dcbc2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:39:52.022361 28275 log.cc:1196] T b799abf0b5b14cfb8070b576480c3eb5 P 1beb07e0229c436dba3ca8ab225dcbc2: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/wal/wals/b799abf0b5b14cfb8070b576480c3eb5
I20260812 16:39:52.022651 28275 ts_tablet_manager.cc:1939] T b799abf0b5b14cfb8070b576480c3eb5 P 1beb07e0229c436dba3ca8ab225dcbc2: Deleting consensus metadata
I20260812 16:39:52.023212 27276 catalog_manager.cc:4841] TS 1beb07e0229c436dba3ca8ab225dcbc2 (127.0.253.195:36289): tablet b799abf0b5b14cfb8070b576480c3eb5 (table test-table [id=71cc5548f2314bdeb7373c133c8b8db4]) successfully deleted
I20260812 16:39:52.023561 28272 tablet_replica.cc:330] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681: stopping tablet replica
I20260812 16:39:52.023708 28272 raft_consensus.cc:2229] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:39:52.023845 28272 raft_consensus.cc:2258] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 16:39:52.026706 27274 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.253.193:44207 (error 108)
I20260812 16:39:52.027262 27672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:52.027122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:44207 (local address 127.0.0.1:50538)
0812 16:39:52.027166 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:52.027218 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.193:44207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260812 16:39:52.027302 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 27669
I20260812 16:39:52.027490 28272 ts_tablet_manager.cc:1905] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 16:39:52.038005 27274 catalog_manager.cc:4584] TS 9503aa07ced9420fbf374cf0b694a872 (127.0.253.193:44207): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b799abf0b5b14cfb8070b576480c3eb5: Network error: Client connection negotiation failed: client connection to 127.0.253.193:44207: connect: Connection refused (error 111)
I20260812 16:39:52.042838 27672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:52.042694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:38999 (local address 127.0.0.1:58474)
0812 16:39:52.042764 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:52.042800 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.194:38999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 16:39:52.042896 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 27940
I20260812 16:39:52.059197 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 27237
I20260812 16:39:52.069862 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:41633
--webserver_interface=127.0.253.254
--webserver_port=33581
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:38235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:41633
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:39:52.153471 28280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:39:52.153680 28280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:39:52.153702 28280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:39:52.154868 28280 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:39:52.154914 28280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:39:52.154928 28280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:39:52.154940 28280 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:39:52.154953 28280 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:39:52.156518 28280 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:38235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:41633
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:41633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=33581
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28280
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:39:52.156791 28280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:39:52.157025 28280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:39:52.162258 28293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:39:52.162230 28294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:39:52.162248 28296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:39:52.162492 28280 server_base.cc:1034] running on GCE node
I20260812 16:39:52.162742 28280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:39:52.163035 28280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:39:52.164228 28280 hybrid_clock.cc:648] HybridClock initialized: now 1786552792164193 us; error 52 us; skew 500 ppm
Aug 12 16:39:52 dist-test-slave-w4v5 krb5kdc[26187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552792, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:39:52.166751 28280 init.cc:377] Logged in from keytab as kudu/127.0.253.254@KRBTEST.COM (short username kudu)
I20260812 16:39:52.167275 28280 webserver.cc:460] Webserver started at http://127.0.253.254:33581/ using document root <none> and password file <none>
I20260812 16:39:52.167487 28280 fs_manager.cc:359] Metadata directory not provided
I20260812 16:39:52.167531 28280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:39:52.168864 28280 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 16:39:52.169561 28309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:52.169819 28280 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 16:39:52.169916 28280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "5c1bd1105c81469c9b3c1cf7dab17ffe"
format_stamp: "Formatted at 2026-08-12 16:39:50 on dist-test-slave-w4v5"
server_key: "b4af129b54d2289074366ee55d0dae69"
server_key_iv: "7822080e3266a7ba3984777f34d0abb6"
server_key_version: "encryptionkey@0"
I20260812 16:39:52.170265 28280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:39:52.194427 28280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:39:52.196518 28280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:39:52.196636 28280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:39:52.201167 28280 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:41633
I20260812 16:39:52.201236 28416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:41633 every 8 connection(s)
I20260812 16:39:52.201643 28280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:39:52.203476 28418 sys_catalog.cc:263] Verifying existing consensus state
I20260812 16:39:52.204248 28418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe: Bootstrap starting.
I20260812 16:39:52.205484 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 28280
I20260812 16:39:52.205833 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.193:44207
--local_ip_for_outbound_sockets=127.0.253.193
--tserver_master_addrs=127.0.253.254:41633
--webserver_port=37137
--webserver_interface=127.0.253.193
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:38235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:39:52.216220 28418 log.cc:824] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe: Log is configured to *not* fsync() on all Append() calls
I20260812 16:39:52.220145 28418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 16:39:52.220566 28418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe: Bootstrap complete.
I20260812 16:39:52.224278 28418 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1bd1105c81469c9b3c1cf7dab17ffe" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 41633 } }
I20260812 16:39:52.224622 28418 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c1bd1105c81469c9b3c1cf7dab17ffe, State: Initialized, Role: FOLLOWER
I20260812 16:39:52.224807 28418 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1bd1105c81469c9b3c1cf7dab17ffe" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 41633 } }
I20260812 16:39:52.224884 28418 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:39:52.224925 28418 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:39:52.224975 28418 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [term 2 FOLLOWER]: Advancing to term 3
I20260812 16:39:52.226392 28418 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1bd1105c81469c9b3c1cf7dab17ffe" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 41633 } }
I20260812 16:39:52.226517 28418 leader_election.cc:302] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c1bd1105c81469c9b3c1cf7dab17ffe; no voters:
I20260812 16:39:52.226727 28418 leader_election.cc:288] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 16:39:52.226820 28434 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [term 3 FOLLOWER]: Leader election won for term 3
I20260812 16:39:52.226986 28434 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [term 3 LEADER]: Becoming Leader. State: Replica: 5c1bd1105c81469c9b3c1cf7dab17ffe, State: Running, Role: LEADER
I20260812 16:39:52.227111 28418 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:39:52.227113 28434 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1bd1105c81469c9b3c1cf7dab17ffe" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 41633 } }
I20260812 16:39:52.227468 28436 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c1bd1105c81469c9b3c1cf7dab17ffe. Latest consensus state: current_term: 3 leader_uuid: "5c1bd1105c81469c9b3c1cf7dab17ffe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1bd1105c81469c9b3c1cf7dab17ffe" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 41633 } } }
I20260812 16:39:52.227563 28436 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [sys.catalog]: This master's current role is: LEADER
I20260812 16:39:52.227465 28435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "5c1bd1105c81469c9b3c1cf7dab17ffe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1bd1105c81469c9b3c1cf7dab17ffe" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 41633 } } }
I20260812 16:39:52.227726 28435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c1bd1105c81469c9b3c1cf7dab17ffe [sys.catalog]: This master's current role is: LEADER
I20260812 16:39:52.227778 28442 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:39:52.228649 28442 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:39:52.228757 28442 catalog_manager.cc:1259] Loaded cluster ID: 4f3572aa723744668178e4247dc42536
I20260812 16:39:52.228801 28442 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260812 16:39:52.229187 28442 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fa076b599ad google::LogMessage::Fail() at ??:0
@ 0x7fa076b5c88a google::LogMessage::SendToLog() at ??:0
@ 0x7fa076b594de google::LogMessage::Flush() at ??:0
@ 0x7fa076b5aa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fa07ea7a0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fa07ea838b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fa078a270d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fa078a1c954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fa0798806db start_thread at ??:0
@ 0x7fa07565761f clone at ??:0
W20260812 16:39:52.326514 28431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:39:52.326805 28431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:39:52.326848 28431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:39:52.328552 28431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:39:52.328619 28431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:39:52.328660 28431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.193
I20260812 16:39:52.330927 28431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:38235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.193:44207
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.253.193
--webserver_port=37137
--tserver_master_addrs=127.0.253.254:41633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.193
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:39:52.331198 28431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:39:52.331454 28431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:39:52.337095 28496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:39:52.337072 28492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:39:52.337059 28493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:39:52.337405 28431 server_base.cc:1034] running on GCE node
I20260812 16:39:52.337623 28431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:39:52.337894 28431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:39:52.339129 28431 hybrid_clock.cc:648] HybridClock initialized: now 1786552792339086 us; error 31 us; skew 500 ppm
Aug 12 16:39:52 dist-test-slave-w4v5 krb5kdc[26187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552792, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:39:52.341725 28431 init.cc:377] Logged in from keytab as kudu/127.0.253.193@KRBTEST.COM (short username kudu)
I20260812 16:39:52.342269 28431 webserver.cc:460] Webserver started at http://127.0.253.193:37137/ using document root <none> and password file <none>
I20260812 16:39:52.342502 28431 fs_manager.cc:359] Metadata directory not provided
I20260812 16:39:52.342576 28431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:39:52.344084 28431 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:39:52.344864 28510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:52.345059 28431 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:39:52.345161 28431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/wal
uuid: "9503aa07ced9420fbf374cf0b694a872"
format_stamp: "Formatted at 2026-08-12 16:39:50 on dist-test-slave-w4v5"
server_key: "2d646c30b1b6ed0c6d8463ad168ff84b"
server_key_iv: "ecadb81110145fef90035ede117b1b14"
server_key_version: "encryptionkey@0"
I20260812 16:39:52.345516 28431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:39:52.376495 28431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:39:52.378945 28431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:39:52.379177 28431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:39:52.379432 28431 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:39:52.379925 28523 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 16:39:52.381971 28431 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 16:39:52.382061 28431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 16:39:52.382115 28431 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 16:39:52.382809 28431 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 16:39:52.382870 28431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 16:39:52.382943 28523 tablet_bootstrap.cc:492] T b799abf0b5b14cfb8070b576480c3eb5 P 9503aa07ced9420fbf374cf0b694a872: Bootstrap starting.
I20260812 16:39:52.390602 28431 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.193:44207
I20260812 16:39:52.390743 28732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.193:44207 every 8 connection(s)
I20260812 16:39:52.391058 28431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
I20260812 16:39:52.393759 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 28431
I20260812 16:39:52.394088 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.194:38999
--local_ip_for_outbound_sockets=127.0.253.194
--tserver_master_addrs=127.0.253.254:41633
--webserver_port=42105
--webserver_interface=127.0.253.194
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:38235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:39:52.394095 28523 log.cc:824] T b799abf0b5b14cfb8070b576480c3eb5 P 9503aa07ced9420fbf374cf0b694a872: Log is configured to *not* fsync() on all Append() calls
I20260812 16:39:52.396801 28523 tablet_bootstrap.cc:492] T b799abf0b5b14cfb8070b576480c3eb5 P 9503aa07ced9420fbf374cf0b694a872: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 16:39:52.397106 28523 tablet_bootstrap.cc:492] T b799abf0b5b14cfb8070b576480c3eb5 P 9503aa07ced9420fbf374cf0b694a872: Bootstrap complete.
I20260812 16:39:52.398134 28523 ts_tablet_manager.cc:1397] T b799abf0b5b14cfb8070b576480c3eb5 P 9503aa07ced9420fbf374cf0b694a872: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.009s
I20260812 16:39:52.399302 28523 raft_consensus.cc:348] T b799abf0b5b14cfb8070b576480c3eb5 P 9503aa07ced9420fbf374cf0b694a872 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beb07e0229c436dba3ca8ab225dcbc2" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 36289 } } peers { permanent_uuid: "2bcb06bb050f462bbe02533992ffb681" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38999 } } peers { permanent_uuid: "9503aa07ced9420fbf374cf0b694a872" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 44207 } }
I20260812 16:39:52.399581 28523 raft_consensus.cc:729] T b799abf0b5b14cfb8070b576480c3eb5 P 9503aa07ced9420fbf374cf0b694a872 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9503aa07ced9420fbf374cf0b694a872, State: Initialized, Role: FOLLOWER
I20260812 16:39:52.399725 28523 consensus_queue.cc:260] T b799abf0b5b14cfb8070b576480c3eb5 P 9503aa07ced9420fbf374cf0b694a872 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beb07e0229c436dba3ca8ab225dcbc2" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 36289 } } peers { permanent_uuid: "2bcb06bb050f462bbe02533992ffb681" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38999 } } peers { permanent_uuid: "9503aa07ced9420fbf374cf0b694a872" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 44207 } }
I20260812 16:39:52.400053 28523 ts_tablet_manager.cc:1428] T b799abf0b5b14cfb8070b576480c3eb5 P 9503aa07ced9420fbf374cf0b694a872: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
W20260812 16:39:52.486197 28740 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:39:52.391920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.193:38581)
0812 16:39:52.392106 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:52.392153 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 16:39:52.392619 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:39:52.485980 (+ 93361us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":104,"threads_started":1}
W20260812 16:39:52.486523 28736 heartbeater.cc:644] Failed to heartbeat to 127.0.253.254:41633 (0 consecutive failures): Network error: Failed to ping master at 127.0.253.254:41633: Client connection negotiation failed: client connection to 127.0.253.254:41633: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 16:39:52.486888 28740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:52.486767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.193:54101)
0812 16:39:52.486822 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:52.486868 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 16:39:52.487335 28740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:52.487211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.193:33189)
0812 16:39:52.487280 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:52.487313 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 16:39:52.487494 28736 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 16:39:52.508419 28743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:39:52.508647 28743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:39:52.508697 28743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:39:52.509831 28743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:39:52.509897 28743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:39:52.509943 28743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.194
I20260812 16:39:52.511814 28743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:38235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.194:38999
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.253.194
--webserver_port=42105
--tserver_master_addrs=127.0.253.254:41633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.194
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:39:52.512095 28743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:39:52.512332 28743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:39:52.517639 28755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:39:52.517778 28752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:39:52.517653 28751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:39:52.517871 28743 server_base.cc:1034] running on GCE node
I20260812 16:39:52.518056 28743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:39:52.518266 28743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:39:52.519407 28743 hybrid_clock.cc:648] HybridClock initialized: now 1786552792519382 us; error 41 us; skew 500 ppm
Aug 12 16:39:52 dist-test-slave-w4v5 krb5kdc[26187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552792, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:39:52.522043 28743 init.cc:377] Logged in from keytab as kudu/127.0.253.194@KRBTEST.COM (short username kudu)
I20260812 16:39:52.522442 28743 webserver.cc:460] Webserver started at http://127.0.253.194:42105/ using document root <none> and password file <none>
I20260812 16:39:52.522709 28743 fs_manager.cc:359] Metadata directory not provided
I20260812 16:39:52.522759 28743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:39:52.524264 28743 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:39:52.524998 28775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:52.525180 28743 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:39:52.525290 28743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/wal
uuid: "2bcb06bb050f462bbe02533992ffb681"
format_stamp: "Formatted at 2026-08-12 16:39:50 on dist-test-slave-w4v5"
server_key: "f334eca05f1bda10a835ce781e2d5e46"
server_key_iv: "a369b104688bd16f17ffd88c8149316d"
server_key_version: "encryptionkey@0"
I20260812 16:39:52.525638 28743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:39:52.534235 28743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:39:52.536595 28743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:39:52.536759 28743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:39:52.536989 28743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:39:52.537436 28790 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 16:39:52.539053 28743 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 16:39:52.539129 28743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 16:39:52.539170 28743 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 16:39:52.539844 28743 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 16:39:52.539877 28743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 16:39:52.539996 28790 tablet_bootstrap.cc:492] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681: Bootstrap starting.
I20260812 16:39:52.541697 28790 log.cc:824] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681: Log is configured to *not* fsync() on all Append() calls
I20260812 16:39:52.545333 28790 tablet_bootstrap.cc:492] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 16:39:52.545661 28790 tablet_bootstrap.cc:492] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681: Bootstrap complete.
I20260812 16:39:52.545861 28790 ts_tablet_manager.cc:1397] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260812 16:39:52.547384 28743 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.194:38999
I20260812 16:39:52.547456 29012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.194:38999 every 8 connection(s)
I20260812 16:39:52.547770 28743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
I20260812 16:39:52.548081 28790 raft_consensus.cc:348] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beb07e0229c436dba3ca8ab225dcbc2" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 36289 } } peers { permanent_uuid: "2bcb06bb050f462bbe02533992ffb681" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38999 } } peers { permanent_uuid: "9503aa07ced9420fbf374cf0b694a872" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 44207 } }
I20260812 16:39:52.548522 28790 raft_consensus.cc:729] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bcb06bb050f462bbe02533992ffb681, State: Initialized, Role: FOLLOWER
I20260812 16:39:52.548692 28790 consensus_queue.cc:260] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beb07e0229c436dba3ca8ab225dcbc2" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 36289 } } peers { permanent_uuid: "2bcb06bb050f462bbe02533992ffb681" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38999 } } peers { permanent_uuid: "9503aa07ced9420fbf374cf0b694a872" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 44207 } }
I20260812 16:39:52.548955 28790 ts_tablet_manager.cc:1428] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681: Time spent starting tablet: real 0.003s user 0.000s sys 0.006s
I20260812 16:39:52.549400 29016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:52.548469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.194:45981)
0812 16:39:52.549121 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:52.549239 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":77,"threads_started":1}
W20260812 16:39:52.549579 29013 heartbeater.cc:644] Failed to heartbeat to 127.0.253.254:41633 (0 consecutive failures): Network error: Failed to ping master at 127.0.253.254:41633: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
I20260812 16:39:52.549865 29016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:52.549765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.194:60777)
0812 16:39:52.549816 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:52.549841 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 16:39:52.550138 29016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:52.550056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.194:60257)
0812 16:39:52.550097 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:52.550120 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260812 16:39:52.550246 29013 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 16:39:52.551645 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 28743
I20260812 16:39:52.551888 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.195:36289
--local_ip_for_outbound_sockets=127.0.253.195
--tserver_master_addrs=127.0.253.254:41633
--webserver_port=45895
--webserver_interface=127.0.253.195
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:38235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:39:52.637251 29018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:39:52.637467 29018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:39:52.637491 29018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:39:52.638662 29018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:39:52.638705 29018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:39:52.638732 29018 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.195
I20260812 16:39:52.640303 29018 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:38235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.195:36289
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.253.195
--webserver_port=45895
--tserver_master_addrs=127.0.253.254:41633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.195
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:39:52.640565 29018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:39:52.640801 29018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:39:52.646103 29025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:39:52.646121 29026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:39:52.646127 29029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:39:52.646430 29018 server_base.cc:1034] running on GCE node
I20260812 16:39:52.646690 29018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:39:52.646888 29018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:39:52.648016 29018 hybrid_clock.cc:648] HybridClock initialized: now 1786552792647997 us; error 37 us; skew 500 ppm
Aug 12 16:39:52 dist-test-slave-w4v5 krb5kdc[26187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552792, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:39:52.650640 29018 init.cc:377] Logged in from keytab as kudu/127.0.253.195@KRBTEST.COM (short username kudu)
I20260812 16:39:52.651114 29018 webserver.cc:460] Webserver started at http://127.0.253.195:45895/ using document root <none> and password file <none>
I20260812 16:39:52.651376 29018 fs_manager.cc:359] Metadata directory not provided
I20260812 16:39:52.651446 29018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:39:52.652878 29018 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:39:52.653540 29041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:52.653713 29018 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:39:52.653820 29018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/wal
uuid: "1beb07e0229c436dba3ca8ab225dcbc2"
format_stamp: "Formatted at 2026-08-12 16:39:50 on dist-test-slave-w4v5"
server_key: "2eb8de141d7e4d20cd6f3732316a1be2"
server_key_iv: "cc52484f739097cf29b3d5e1a3d4517b"
server_key_version: "encryptionkey@0"
I20260812 16:39:52.654174 29018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:39:52.677397 29018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:39:52.679502 29018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:39:52.679673 29018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:39:52.679975 29018 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:39:52.680310 29018 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:39:52.680364 29018 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:52.680400 29018 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:39:52.680444 29018 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:39:52.686384 29018 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.195:36289
I20260812 16:39:52.686443 29274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.195:36289 every 8 connection(s)
I20260812 16:39:52.686825 29018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
I20260812 16:39:52.687978 29280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:52.687542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.195:44827)
0812 16:39:52.687740 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:52.687808 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":112,"threads_started":1}
I20260812 16:39:52.688167 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 29018
W20260812 16:39:52.688478 29276 heartbeater.cc:644] Failed to heartbeat to 127.0.253.254:41633 (0 consecutive failures): Network error: Failed to ping master at 127.0.253.254:41633: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
I20260812 16:39:52.688804 29280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:52.688703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.195:51609)
0812 16:39:52.688757 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:52.688783 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 16:39:52.689091 29280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:52.689001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.195:54893)
0812 16:39:52.689050 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:52.689073 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260812 16:39:52.689201 29276 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 16:39:53.488195 29286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:53.487946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.193:57885)
0812 16:39:53.488081 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:53.488122 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":69,"threads_started":1}
I20260812 16:39:53.550745 29288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:53.550561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.194:58067)
0812 16:39:53.550675 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:53.550710 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":57,"threads_started":1}
I20260812 16:39:53.689960 29289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:53.689496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.195:60363)
0812 16:39:53.689876 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:53.689908 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":316,"threads_started":1}
I20260812 16:39:53.816815 29291 raft_consensus.cc:482] T b799abf0b5b14cfb8070b576480c3eb5 P 9503aa07ced9420fbf374cf0b694a872 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:39:53.816960 29291 raft_consensus.cc:504] T b799abf0b5b14cfb8070b576480c3eb5 P 9503aa07ced9420fbf374cf0b694a872 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beb07e0229c436dba3ca8ab225dcbc2" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 36289 } } peers { permanent_uuid: "2bcb06bb050f462bbe02533992ffb681" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38999 } } peers { permanent_uuid: "9503aa07ced9420fbf374cf0b694a872" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 44207 } }
I20260812 16:39:53.817386 29291 leader_election.cc:288] T b799abf0b5b14cfb8070b576480c3eb5 P 9503aa07ced9420fbf374cf0b694a872 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1beb07e0229c436dba3ca8ab225dcbc2 (127.0.253.195:36289), 2bcb06bb050f462bbe02533992ffb681 (127.0.253.194:38999)
Aug 12 16:39:53 dist-test-slave-w4v5 krb5kdc[26187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552792, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for kudu/127.0.253.195@KRBTEST.COM
Aug 12 16:39:53 dist-test-slave-w4v5 krb5kdc[26187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552792, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for kudu/127.0.253.194@KRBTEST.COM
I20260812 16:39:53.830209 29286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:53.817596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.195:36289 (local address 127.0.253.193:49331)
0812 16:39:53.817652 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:53.817666 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 16:39:53.817745 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:39:53.818605 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:39:53.818605 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:39:53.818876 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:39:53.819191 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:53.819199 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:53.820935 (+ 1736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:53.820937 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:53.822251 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:53.822254 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:53.823133 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:53.823134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:53.823298 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:53.823987 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:39:53.823999 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:39:53.826093 (+ 2094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:39:53.828474 (+ 2381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:53.828483 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:53.828486 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:53.828810 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:53.829044 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:53.829045 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:53.829046 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:53.829216 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:53.829631 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:39:53.829638 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:39:53.829803 (+ 165us) client_negotiation.cc:769] Sending connection context
0812 16:39:53.829913 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 16:39:53.830096 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":49}
I20260812 16:39:53.830225 29294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:53.817744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:49331 (local address 127.0.253.195:36289)
0812 16:39:53.817988 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:53.817997 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:53.818023 (+ 26us) server_negotiation.cc:407] Connection header received
0812 16:39:53.818452 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:53.818454 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:53.818488 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:53.818600 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:39:53.819350 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:53.820803 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:53.822355 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:53.822990 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:53.823138 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:53.826224 (+ 3086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:39:53.826249 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:39:53.826249 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:39:53.826265 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:39:53.828374 (+ 2109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:53.828895 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:53.828897 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:53.828897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:53.828989 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:53.829302 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:53.829302 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:53.829303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:53.829564 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:39:53.829652 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:53.829980 (+ 328us) server_negotiation.cc:299] Negotiation successful
0812 16:39:53.830108 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":62,"threads_started":1}
I20260812 16:39:53.830649 29292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:53.817648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:38999 (local address 127.0.253.193:59167)
0812 16:39:53.817773 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:53.817789 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 16:39:53.817847 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:39:53.818392 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:39:53.818395 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:39:53.818888 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:39:53.819191 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:53.819199 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:53.820950 (+ 1751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:53.820951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:53.822251 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:53.822254 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:53.823173 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:53.823174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:53.823298 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:53.823995 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:39:53.823999 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:39:53.826496 (+ 2497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:39:53.829366 (+ 2870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:53.829367 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:53.829367 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:53.829544 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:53.829815 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:53.829816 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:53.829816 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:53.829883 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:53.830280 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:39:53.830281 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:39:53.830331 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 16:39:53.830420 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 16:39:53.830541 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":88,"thread_start_us":72,"threads_started":1}
I20260812 16:39:53.830776 29295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:53.817806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:59167 (local address 127.0.253.194:38999)
0812 16:39:53.817968 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:53.817979 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:53.818000 (+ 21us) server_negotiation.cc:407] Connection header received
0812 16:39:53.818116 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:53.818118 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:53.818160 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:53.818324 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:39:53.819341 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:53.820804 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:53.822355 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:53.823018 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:53.823174 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:53.826613 (+ 3439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:39:53.826622 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:39:53.826623 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:39:53.826634 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:39:53.829293 (+ 2659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:53.829631 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:53.829634 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:53.829634 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:53.829733 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:53.829952 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:53.829953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:53.829954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:53.830188 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:39:53.830288 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:53.830497 (+ 209us) server_negotiation.cc:299] Negotiation successful
0812 16:39:53.830630 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":64,"threads_started":1}
I20260812 16:39:53.831213 28918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b799abf0b5b14cfb8070b576480c3eb5" candidate_uuid: "9503aa07ced9420fbf374cf0b694a872" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "2bcb06bb050f462bbe02533992ffb681" is_pre_election: true
I20260812 16:39:53.831360 28918 raft_consensus.cc:2454] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9503aa07ced9420fbf374cf0b694a872 in term 1.
I20260812 16:39:53.831642 28512 leader_election.cc:302] T b799abf0b5b14cfb8070b576480c3eb5 P 9503aa07ced9420fbf374cf0b694a872 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2bcb06bb050f462bbe02533992ffb681, 9503aa07ced9420fbf374cf0b694a872; no voters:
I20260812 16:39:53.831815 29291 raft_consensus.cc:2781] T b799abf0b5b14cfb8070b576480c3eb5 P 9503aa07ced9420fbf374cf0b694a872 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 16:39:53.831888 29291 raft_consensus.cc:482] T b799abf0b5b14cfb8070b576480c3eb5 P 9503aa07ced9420fbf374cf0b694a872 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:39:53.831916 29291 raft_consensus.cc:3037] T b799abf0b5b14cfb8070b576480c3eb5 P 9503aa07ced9420fbf374cf0b694a872 [term 1 FOLLOWER]: Advancing to term 2
W20260812 16:39:53.831969 28517 leader_election.cc:341] T b799abf0b5b14cfb8070b576480c3eb5 P 9503aa07ced9420fbf374cf0b694a872 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1beb07e0229c436dba3ca8ab225dcbc2 (127.0.253.195:36289): Not found: Tablet not found: b799abf0b5b14cfb8070b576480c3eb5
I20260812 16:39:53.833098 29291 raft_consensus.cc:504] T b799abf0b5b14cfb8070b576480c3eb5 P 9503aa07ced9420fbf374cf0b694a872 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beb07e0229c436dba3ca8ab225dcbc2" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 36289 } } peers { permanent_uuid: "2bcb06bb050f462bbe02533992ffb681" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38999 } } peers { permanent_uuid: "9503aa07ced9420fbf374cf0b694a872" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 44207 } }
I20260812 16:39:53.833271 29291 leader_election.cc:288] T b799abf0b5b14cfb8070b576480c3eb5 P 9503aa07ced9420fbf374cf0b694a872 [CANDIDATE]: Term 2 election: Requested vote from peers 1beb07e0229c436dba3ca8ab225dcbc2 (127.0.253.195:36289), 2bcb06bb050f462bbe02533992ffb681 (127.0.253.194:38999)
I20260812 16:39:53.833506 28918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b799abf0b5b14cfb8070b576480c3eb5" candidate_uuid: "9503aa07ced9420fbf374cf0b694a872" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "2bcb06bb050f462bbe02533992ffb681"
I20260812 16:39:53.833611 28918 raft_consensus.cc:3037] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681 [term 1 FOLLOWER]: Advancing to term 2
W20260812 16:39:53.833691 28517 leader_election.cc:341] T b799abf0b5b14cfb8070b576480c3eb5 P 9503aa07ced9420fbf374cf0b694a872 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 1beb07e0229c436dba3ca8ab225dcbc2 (127.0.253.195:36289): Not found: Tablet not found: b799abf0b5b14cfb8070b576480c3eb5
I20260812 16:39:53.834834 28918 raft_consensus.cc:2454] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9503aa07ced9420fbf374cf0b694a872 in term 2.
I20260812 16:39:53.835023 28512 leader_election.cc:302] T b799abf0b5b14cfb8070b576480c3eb5 P 9503aa07ced9420fbf374cf0b694a872 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2bcb06bb050f462bbe02533992ffb681, 9503aa07ced9420fbf374cf0b694a872; no voters: 1beb07e0229c436dba3ca8ab225dcbc2
I20260812 16:39:53.835157 29291 raft_consensus.cc:2781] T b799abf0b5b14cfb8070b576480c3eb5 P 9503aa07ced9420fbf374cf0b694a872 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 16:39:53.835278 29291 raft_consensus.cc:686] T b799abf0b5b14cfb8070b576480c3eb5 P 9503aa07ced9420fbf374cf0b694a872 [term 2 LEADER]: Becoming Leader. State: Replica: 9503aa07ced9420fbf374cf0b694a872, State: Running, Role: LEADER
I20260812 16:39:53.835404 29291 consensus_queue.cc:237] T b799abf0b5b14cfb8070b576480c3eb5 P 9503aa07ced9420fbf374cf0b694a872 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beb07e0229c436dba3ca8ab225dcbc2" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 36289 } } peers { permanent_uuid: "2bcb06bb050f462bbe02533992ffb681" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38999 } } peers { permanent_uuid: "9503aa07ced9420fbf374cf0b694a872" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 44207 } }
I20260812 16:39:53.835872 29286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:53.835616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.193:47041)
0812 16:39:53.835813 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:53.835839 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179}
W20260812 16:39:54.220938 28517 consensus_peers.cc:487] T b799abf0b5b14cfb8070b576480c3eb5 P 9503aa07ced9420fbf374cf0b694a872 -> Peer 1beb07e0229c436dba3ca8ab225dcbc2 (127.0.253.195:36289): Couldn't send request to peer 1beb07e0229c436dba3ca8ab225dcbc2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b799abf0b5b14cfb8070b576480c3eb5. This is attempt 1: this message will repeat every 5th retry.
I20260812 16:39:54.225543 28918 raft_consensus.cc:1261] T b799abf0b5b14cfb8070b576480c3eb5 P 2bcb06bb050f462bbe02533992ffb681 [term 2 FOLLOWER]: Refusing update from remote peer 9503aa07ced9420fbf374cf0b694a872: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260812 16:39:54.225956 29291 consensus_queue.cc:1035] T b799abf0b5b14cfb8070b576480c3eb5 P 9503aa07ced9420fbf374cf0b694a872 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bcb06bb050f462bbe02533992ffb681" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 16:39:54.226991 29311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:54.226812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.194:44729)
0812 16:39:54.226893 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:54.226942 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65,"thread_start_us":48,"threads_started":1}
I20260812 16:39:54.227922 29286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:54.227810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.193:44797)
0812 16:39:54.227858 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:54.227886 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260812 16:39:54.227973 29311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:54.227879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.194:42143)
0812 16:39:54.227915 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:54.227945 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260812 16:39:54.690610 29315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:54.690395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.195:43775)
0812 16:39:54.690534 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:54.690573 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":74,"threads_started":1}
I20260812 16:39:54.812501 29319 ts_tablet_manager.cc:927] T b799abf0b5b14cfb8070b576480c3eb5 P 1beb07e0229c436dba3ca8ab225dcbc2: Initiating tablet copy from peer 9503aa07ced9420fbf374cf0b694a872 (127.0.253.193:44207)
I20260812 16:39:54.812747 29319 tablet_copy_client.cc:316] T b799abf0b5b14cfb8070b576480c3eb5 P 1beb07e0229c436dba3ca8ab225dcbc2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.253.193:44207
Aug 12 16:39:54 dist-test-slave-w4v5 krb5kdc[26187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552792, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for kudu/127.0.253.193@KRBTEST.COM
I20260812 16:39:54.821491 29323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:54.813164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:58185 (local address 127.0.253.193:44207)
0812 16:39:54.813299 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 16:39:54.813305 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:39:54.813315 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:39:54.813341 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:39:54.813343 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:39:54.813356 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:39:54.813421 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:39:54.813910 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:54.814590 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:54.815520 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:39:54.815911 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:39:54.815983 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:54.818359 (+ 2376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:39:54.818370 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:39:54.818371 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:39:54.818387 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:39:54.820084 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:54.820535 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:54.820539 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:54.820539 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:54.820607 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:39:54.820827 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:39:54.820829 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:39:54.820829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:39:54.820990 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:39:54.821085 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 16:39:54.821292 (+ 207us) server_negotiation.cc:299] Negotiation successful
0812 16:39:54.821386 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":66,"threads_started":1}
I20260812 16:39:54.821504 29315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:54.813045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:44207 (local address 127.0.253.195:58185)
0812 16:39:54.813119 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:54.813148 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0812 16:39:54.813245 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:39:54.813439 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:39:54.813446 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:39:54.813745 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:39:54.813826 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:54.813828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:54.814690 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:54.814691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:54.815421 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:39:54.815424 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:39:54.816015 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:39:54.816016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:39:54.816095 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:39:54.816714 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:39:54.816719 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:39:54.818248 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:39:54.820191 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:54.820200 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:54.820202 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:54.820441 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:54.820689 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:39:54.820690 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:39:54.820690 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:39:54.820760 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:39:54.821075 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:39:54.821083 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:39:54.821145 (+ 62us) client_negotiation.cc:769] Sending connection context
0812 16:39:54.821255 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 16:39:54.821389 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 16:39:54.822037 28682 tablet_copy_service.cc:140] P 9503aa07ced9420fbf374cf0b694a872: Received BeginTabletCopySession request for tablet b799abf0b5b14cfb8070b576480c3eb5 from peer 1beb07e0229c436dba3ca8ab225dcbc2 ({username='kudu', principal='kudu/127.0.253.195@KRBTEST.COM'} at 127.0.253.195:58185)
I20260812 16:39:54.822113 28682 tablet_copy_service.cc:161] P 9503aa07ced9420fbf374cf0b694a872: Beginning new tablet copy session on tablet b799abf0b5b14cfb8070b576480c3eb5 from peer 1beb07e0229c436dba3ca8ab225dcbc2 at {username='kudu', principal='kudu/127.0.253.195@KRBTEST.COM'} at 127.0.253.195:58185: session id = 1beb07e0229c436dba3ca8ab225dcbc2-b799abf0b5b14cfb8070b576480c3eb5
I20260812 16:39:54.822654 28682 tablet_copy_source_session.cc:205] T b799abf0b5b14cfb8070b576480c3eb5 P 9503aa07ced9420fbf374cf0b694a872: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 16:39:54.823714 29319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b799abf0b5b14cfb8070b576480c3eb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:39:54.826399 29319 tablet_copy_client.cc:791] T b799abf0b5b14cfb8070b576480c3eb5 P 1beb07e0229c436dba3ca8ab225dcbc2: tablet copy: Starting download of 0 data blocks...
I20260812 16:39:54.826575 29319 tablet_copy_client.cc:655] T b799abf0b5b14cfb8070b576480c3eb5 P 1beb07e0229c436dba3ca8ab225dcbc2: tablet copy: Starting download of 1 WAL segments...
I20260812 16:39:54.827596 29319 tablet_copy_client.cc:531] T b799abf0b5b14cfb8070b576480c3eb5 P 1beb07e0229c436dba3ca8ab225dcbc2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 16:39:54.828385 29319 tablet_bootstrap.cc:492] T b799abf0b5b14cfb8070b576480c3eb5 P 1beb07e0229c436dba3ca8ab225dcbc2: Bootstrap starting.
I20260812 16:39:54.829873 29319 log.cc:824] T b799abf0b5b14cfb8070b576480c3eb5 P 1beb07e0229c436dba3ca8ab225dcbc2: Log is configured to *not* fsync() on all Append() calls
I20260812 16:39:54.832515 29319 tablet_bootstrap.cc:492] T b799abf0b5b14cfb8070b576480c3eb5 P 1beb07e0229c436dba3ca8ab225dcbc2: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 16:39:54.832762 29319 tablet_bootstrap.cc:492] T b799abf0b5b14cfb8070b576480c3eb5 P 1beb07e0229c436dba3ca8ab225dcbc2: Bootstrap complete.
I20260812 16:39:54.832914 29319 ts_tablet_manager.cc:1397] T b799abf0b5b14cfb8070b576480c3eb5 P 1beb07e0229c436dba3ca8ab225dcbc2: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 16:39:54.834074 29319 raft_consensus.cc:348] T b799abf0b5b14cfb8070b576480c3eb5 P 1beb07e0229c436dba3ca8ab225dcbc2 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beb07e0229c436dba3ca8ab225dcbc2" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 36289 } } peers { permanent_uuid: "2bcb06bb050f462bbe02533992ffb681" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38999 } } peers { permanent_uuid: "9503aa07ced9420fbf374cf0b694a872" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 44207 } }
I20260812 16:39:54.834264 29319 raft_consensus.cc:729] T b799abf0b5b14cfb8070b576480c3eb5 P 1beb07e0229c436dba3ca8ab225dcbc2 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1beb07e0229c436dba3ca8ab225dcbc2, State: Initialized, Role: FOLLOWER
I20260812 16:39:54.834408 29319 consensus_queue.cc:260] T b799abf0b5b14cfb8070b576480c3eb5 P 1beb07e0229c436dba3ca8ab225dcbc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beb07e0229c436dba3ca8ab225dcbc2" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 36289 } } peers { permanent_uuid: "2bcb06bb050f462bbe02533992ffb681" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 38999 } } peers { permanent_uuid: "9503aa07ced9420fbf374cf0b694a872" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 44207 } }
I20260812 16:39:54.834676 29319 ts_tablet_manager.cc:1428] T b799abf0b5b14cfb8070b576480c3eb5 P 1beb07e0229c436dba3ca8ab225dcbc2: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 16:39:54.835045 29315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:54.834932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.195:45547)
0812 16:39:54.834991 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:54.835022 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 16:39:54.835167 28682 tablet_copy_service.cc:342] P 9503aa07ced9420fbf374cf0b694a872: Request end of tablet copy session 1beb07e0229c436dba3ca8ab225dcbc2-b799abf0b5b14cfb8070b576480c3eb5 received from {username='kudu', principal='kudu/127.0.253.195@KRBTEST.COM'} at 127.0.253.195:58185
I20260812 16:39:54.835240 28682 tablet_copy_service.cc:433] P 9503aa07ced9420fbf374cf0b694a872: ending tablet copy session 1beb07e0229c436dba3ca8ab225dcbc2-b799abf0b5b14cfb8070b576480c3eb5 on tablet b799abf0b5b14cfb8070b576480c3eb5 with peer 1beb07e0229c436dba3ca8ab225dcbc2
I20260812 16:39:54.835737 29315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:54.835625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.195:49617)
0812 16:39:54.835673 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:54.835704 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 16:39:55.228593 29331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:55.228371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.194:49815)
0812 16:39:55.228497 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:55.228538 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":52,"threads_started":1}
I20260812 16:39:55.228642 29332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:55.228381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.193:40479)
0812 16:39:55.228564 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:55.228600 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":111,"threads_started":1}
I20260812 16:39:55.301046 29332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:55.300865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.193:54005)
0812 16:39:55.300939 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:55.300992 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260812 16:39:55.301046 29191 raft_consensus.cc:1203] T b799abf0b5b14cfb8070b576480c3eb5 P 1beb07e0229c436dba3ca8ab225dcbc2 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260812 16:39:55.301586 29315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:55.301480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.195:34995)
0812 16:39:55.301519 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:55.301558 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260812 16:39:56.229434 29339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:56.229088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.194:50255)
0812 16:39:56.229311 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:56.229352 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":102,"threads_started":1}
I20260812 16:39:56.301860 29341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:56.301547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.193:49029)
0812 16:39:56.301721 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:56.301766 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":83,"threads_started":1}
I20260812 16:39:56.302412 29342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:56.302200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.195:36035)
0812 16:39:56.302337 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:56.302372 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":64,"threads_started":1}
I20260812 16:39:57.230275 29344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:57.230012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.194:45919)
0812 16:39:57.230179 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:57.230226 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":91,"threads_started":1}
I20260812 16:39:57.302738 29346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:57.302478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.193:58501)
0812 16:39:57.302641 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:57.302685 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":67,"threads_started":1}
I20260812 16:39:57.303160 29347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:57.302842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.195:36879)
0812 16:39:57.303020 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:57.303059 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":92,"threads_started":1}
I20260812 16:39:58.231199 29348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:58.230894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.194:54589)
0812 16:39:58.231039 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:58.231138 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":70,"threads_started":1}
I20260812 16:39:58.303660 29352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:58.303389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.193:34475)
0812 16:39:58.303556 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:58.303612 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":86,"threads_started":1}
I20260812 16:39:58.303929 29353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:58.303668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.195:41959)
0812 16:39:58.303836 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:58.303876 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":103,"threads_started":1}
I20260812 16:39:58.304098 29352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:58.303944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.193:39501)
0812 16:39:58.304026 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:58.304063 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260812 16:39:59.232026 29354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:59.231754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.194:39583)
0812 16:39:59.231932 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:59.231975 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":73,"threads_started":1}
I20260812 16:39:59.304626 29355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:59.304397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.195:53255)
0812 16:39:59.304546 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:59.304579 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":88,"threads_started":1}
I20260812 16:39:59.304854 29356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:39:59.304615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.193:57755)
0812 16:39:59.304747 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0812 16:39:59.304787 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":80,"threads_started":1}
I20260812 16:40:00.232846 29357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:00.232564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.194:56935)
0812 16:40:00.232739 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:00.232782 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":106,"threads_started":1}
I20260812 16:40:00.305572 29359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:00.305311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.193:59487)
0812 16:40:00.305485 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:00.305520 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":84,"threads_started":1}
I20260812 16:40:00.305588 29360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:00.305306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.195:38043)
0812 16:40:00.305463 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:00.305546 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":85,"threads_started":1}
I20260812 16:40:01.233687 29361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:01.233413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.194:52859)
0812 16:40:01.233589 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:01.233637 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":83,"threads_started":1}
I20260812 16:40:01.306466 29366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:01.306205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.195:39087)
0812 16:40:01.306359 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:01.306407 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":84,"threads_started":1}
I20260812 16:40:01.306466 29368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:01.306205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.193:37701)
0812 16:40:01.306387 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:01.306422 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":86,"threads_started":1}
I20260812 16:40:02.234540 29371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:02.234267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.194:44151)
0812 16:40:02.234431 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:02.234480 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":73,"threads_started":1}
I20260812 16:40:02.307367 29373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:02.307129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.195:57105)
0812 16:40:02.307279 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:02.307319 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":80,"threads_started":1}
I20260812 16:40:02.307368 29374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:02.307143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.193:37025)
0812 16:40:02.307283 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:02.307317 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":65,"threads_started":1}
I20260812 16:40:03.235409 29375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:03.235157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.194:56711)
0812 16:40:03.235313 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:03.235360 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":80,"threads_started":1}
I20260812 16:40:03.308082 29382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:03.307829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.195:50321)
0812 16:40:03.307989 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:03.308029 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":78,"threads_started":1}
I20260812 16:40:03.308116 29385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:03.307876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.193:37765)
0812 16:40:03.308027 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:03.308064 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":86,"threads_started":1}
I20260812 16:40:04.236200 29386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:04.235931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.194:56167)
0812 16:40:04.236092 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:04.236141 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":98,"threads_started":1}
I20260812 16:40:04.308812 29396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:04.308580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.193:38421)
0812 16:40:04.308741 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:04.308769 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":90,"threads_started":1}
I20260812 16:40:04.308810 29391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:04.308582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.195:40009)
0812 16:40:04.308729 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:04.308763 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":87,"threads_started":1}
I20260812 16:40:05.236929 29400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:05.236672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.194:58853)
0812 16:40:05.236837 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:05.236879 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":72,"threads_started":1}
I20260812 16:40:05.309672 29402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:05.309386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.195:48635)
0812 16:40:05.309569 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:05.309613 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":66,"threads_started":1}
I20260812 16:40:05.309710 29412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:05.309396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.193:35483)
0812 16:40:05.309568 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:05.309613 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":73,"threads_started":1}
I20260812 16:40:06.237751 29414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:06.237506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.194:55261)
0812 16:40:06.237657 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:06.237701 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":68,"threads_started":1}
I20260812 16:40:06.310526 29416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:06.310296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.193:49849)
0812 16:40:06.310430 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:06.310458 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":60,"threads_started":1}
I20260812 16:40:06.310662 29418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:06.310404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.195:37815)
0812 16:40:06.310554 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:06.310608 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":67,"threads_started":1}
I20260812 16:40:07.238569 29420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:07.238292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.194:45183)
0812 16:40:07.238469 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:07.238518 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":71,"threads_started":1}
I20260812 16:40:07.311326 29422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:07.311010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.193:57189)
0812 16:40:07.311213 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:07.311259 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":123,"threads_started":1}
I20260812 16:40:07.311445 29425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:07.311184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41633 (local address 127.0.253.195:52903)
0812 16:40:07.311344 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:07.311389 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":69,"threads_started":1}
I20260812 16:40:07.690021 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 28431
I20260812 16:40:07.709131 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 28743
I20260812 16:40:07.734715 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 29018
2026-08-12T16:40:07Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18942 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:40:07 dist-test-slave-w4v5 krb5kdc[29431](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:40:07 dist-test-slave-w4v5 krb5kdc[29431](info): set up 2 sockets
Aug 12 16:40:07 dist-test-slave-w4v5 krb5kdc[29431](info): commencing operation
krb5kdc: starting...
W20260812 16:40:09.095270 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.329s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:40:09 dist-test-slave-w4v5 krb5kdc[29431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552809, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:40:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:40:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
I20260812 16:40:09.228436 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:39981
--webserver_interface=127.0.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:43519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:39981
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:40:09.314706 29457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:40:09.314922 29457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:40:09.314945 29457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:40:09.316118 29457 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 16:40:09.316161 29457 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 16:40:09.316175 29457 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 16:40:09.316186 29457 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:40:09.316200 29457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:40:09.316213 29457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:40:09.316231 29457 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:40:09.316243 29457 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:40:09.317688 29457 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:43519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:39981
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:39981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:40:09.317888 29457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:40:09.318056 29457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:40:09.323658 29473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:40:09.323684 29457 server_base.cc:1034] running on GCE node
W20260812 16:40:09.323726 29469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:40:09.323658 29468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:40:09.324059 29457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:40:09.324409 29457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:40:09.325562 29457 hybrid_clock.cc:648] HybridClock initialized: now 1786552809325542 us; error 59 us; skew 500 ppm
Aug 12 16:40:09 dist-test-slave-w4v5 krb5kdc[29431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552809, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:40:09.328130 29457 init.cc:377] Logged in from keytab as kudu/127.0.253.254@KRBTEST.COM (short username kudu)
I20260812 16:40:09.328621 29457 webserver.cc:460] Webserver started at http://127.0.253.254:33263/ using document root <none> and password file <none>
I20260812 16:40:09.328867 29457 fs_manager.cc:359] Metadata directory not provided
I20260812 16:40:09.328940 29457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:40:09.329101 29457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:40:09.330199 29457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "3bddd07e41ca4d92aaf62a0b496011ea"
format_stamp: "Formatted at 2026-08-12 16:40:09 on dist-test-slave-w4v5"
server_key: "ba0b99e8997ca623764cf5d495c46212"
server_key_iv: "ae6fc5547a948212895046dc3e8bed98"
server_key_version: "encryptionkey@0"
I20260812 16:40:09.330601 29457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "3bddd07e41ca4d92aaf62a0b496011ea"
format_stamp: "Formatted at 2026-08-12 16:40:09 on dist-test-slave-w4v5"
server_key: "ba0b99e8997ca623764cf5d495c46212"
server_key_iv: "ae6fc5547a948212895046dc3e8bed98"
server_key_version: "encryptionkey@0"
I20260812 16:40:09.332089 29457 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:40:09.333160 29484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:40:09.333369 29457 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:40:09.333477 29457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "3bddd07e41ca4d92aaf62a0b496011ea"
format_stamp: "Formatted at 2026-08-12 16:40:09 on dist-test-slave-w4v5"
server_key: "ba0b99e8997ca623764cf5d495c46212"
server_key_iv: "ae6fc5547a948212895046dc3e8bed98"
server_key_version: "encryptionkey@0"
I20260812 16:40:09.333570 29457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:40:09.360944 29457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:40:09.363715 29457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:40:09.363904 29457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:40:09.368316 29457 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:39981
I20260812 16:40:09.368350 29585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:39981 every 8 connection(s)
I20260812 16:40:09.368723 29457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:40:09.369721 29586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:40:09.372138 29586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea: Bootstrap starting.
I20260812 16:40:09.372727 29586 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea: Neither blocks nor log segments found. Creating new log.
I20260812 16:40:09.372975 29586 log.cc:824] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea: Log is configured to *not* fsync() on all Append() calls
I20260812 16:40:09.373732 29586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea: No bootstrap required, opened a new log
I20260812 16:40:09.375113 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 29457
I20260812 16:40:09.375218 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:40:09.375404 1015 external_mini_cluster.cc:1428] Setting key 9021b3c2b3568c095c66dffebfee4838
I20260812 16:40:09.375746 29586 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bddd07e41ca4d92aaf62a0b496011ea" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39981 } }
I20260812 16:40:09.375942 29586 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:40:09.375996 29586 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bddd07e41ca4d92aaf62a0b496011ea, State: Initialized, Role: FOLLOWER
I20260812 16:40:09.376163 29586 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bddd07e41ca4d92aaf62a0b496011ea" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39981 } }
I20260812 16:40:09.376263 29586 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:40:09.376299 29586 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:40:09.376353 29586 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:40:09.377521 29586 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bddd07e41ca4d92aaf62a0b496011ea" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39981 } }
I20260812 16:40:09.377615 29586 leader_election.cc:302] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3bddd07e41ca4d92aaf62a0b496011ea; no voters:
I20260812 16:40:09.377801 29586 leader_election.cc:288] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:40:09.377833 29610 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:40:09.377950 29610 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [term 1 LEADER]: Becoming Leader. State: Replica: 3bddd07e41ca4d92aaf62a0b496011ea, State: Running, Role: LEADER
I20260812 16:40:09.378074 29586 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:40:09.378137 29610 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bddd07e41ca4d92aaf62a0b496011ea" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39981 } }
I20260812 16:40:09.378599 29610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3bddd07e41ca4d92aaf62a0b496011ea. Latest consensus state: current_term: 1 leader_uuid: "3bddd07e41ca4d92aaf62a0b496011ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bddd07e41ca4d92aaf62a0b496011ea" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39981 } } }
I20260812 16:40:09.378692 29612 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3bddd07e41ca4d92aaf62a0b496011ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bddd07e41ca4d92aaf62a0b496011ea" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39981 } } }
I20260812 16:40:09.378746 29610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [sys.catalog]: This master's current role is: LEADER
I20260812 16:40:09.378783 29612 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [sys.catalog]: This master's current role is: LEADER
I20260812 16:40:09.378995 29619 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:40:09.379611 29619 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:40:09.381692 29619 catalog_manager.cc:1347] Generated new cluster ID: 7580f6e5149941e8865a4353b78d6187
I20260812 16:40:09.381758 29619 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 16:40:09 dist-test-slave-w4v5 krb5kdc[29431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552809, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:40:09.387192 29609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:09.375964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55712 (local address 127.0.253.254:39981)
0812 16:40:09.376193 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0812 16:40:09.376202 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:40:09.376239 (+ 37us) server_negotiation.cc:407] Connection header received
0812 16:40:09.376392 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:40:09.376400 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:40:09.376459 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:40:09.376571 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:40:09.377433 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:09.378781 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:09.380043 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:09.380510 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:09.380649 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:09.383528 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:40:09.383532 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:40:09.383533 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:40:09.383544 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:40:09.385459 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:40:09.385884 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:40:09.385887 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:40:09.385887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:40:09.386007 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:40:09.386240 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:40:09.386241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:40:09.386241 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:40:09.386523 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:40:09.386663 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 16:40:09.386842 (+ 179us) server_negotiation.cc:299] Negotiation successful
0812 16:40:09.386975 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":94,"threads_started":1}
I20260812 16:40:09.396044 29619 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:40:09.396776 29619 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:40:09.401629 29619 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea: Generated new TSK 0
I20260812 16:40:09.401836 29619 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab.
Entry for principal kudu/127.0.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab.
I20260812 16:40:09.429597 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.193:0
--local_ip_for_outbound_sockets=127.0.253.193
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:39981
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:43519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:40:09.515411 29644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:40:09.515659 29644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:40:09.515699 29644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:40:09.516842 29644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:40:09.516914 29644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:40:09.516959 29644 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.193
I20260812 16:40:09.518448 29644 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:43519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:39981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.193
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:40:09.518723 29644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:40:09.518962 29644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 16:40:09.524437 29644 server_base.cc:1034] running on GCE node
W20260812 16:40:09.524408 29665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:40:09.524399 29661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:40:09.524394 29660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:40:09.524866 29644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:40:09.525157 29644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:40:09.526350 29644 hybrid_clock.cc:648] HybridClock initialized: now 1786552809526334 us; error 48 us; skew 500 ppm
Aug 12 16:40:09 dist-test-slave-w4v5 krb5kdc[29431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552809, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:40:09.529204 29644 init.cc:377] Logged in from keytab as kudu/127.0.253.193@KRBTEST.COM (short username kudu)
I20260812 16:40:09.529644 29644 webserver.cc:460] Webserver started at http://127.0.253.193:46433/ using document root <none> and password file <none>
I20260812 16:40:09.529891 29644 fs_manager.cc:359] Metadata directory not provided
I20260812 16:40:09.529964 29644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:40:09.530117 29644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:40:09.531286 29644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/data/instance:
uuid: "0e7940e58fe64aa3a248e733e4e5ccea"
format_stamp: "Formatted at 2026-08-12 16:40:09 on dist-test-slave-w4v5"
server_key: "277a689f10147fc0e9b471a980e66894"
server_key_iv: "d58bb779abd4e0042bb7dca4fff0fa46"
server_key_version: "encryptionkey@0"
I20260812 16:40:09.531663 29644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance:
uuid: "0e7940e58fe64aa3a248e733e4e5ccea"
format_stamp: "Formatted at 2026-08-12 16:40:09 on dist-test-slave-w4v5"
server_key: "277a689f10147fc0e9b471a980e66894"
server_key_iv: "d58bb779abd4e0042bb7dca4fff0fa46"
server_key_version: "encryptionkey@0"
I20260812 16:40:09.533054 29644 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:40:09.534101 29680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:40:09.534322 29644 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:40:09.534438 29644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/wal
uuid: "0e7940e58fe64aa3a248e733e4e5ccea"
format_stamp: "Formatted at 2026-08-12 16:40:09 on dist-test-slave-w4v5"
server_key: "277a689f10147fc0e9b471a980e66894"
server_key_iv: "d58bb779abd4e0042bb7dca4fff0fa46"
server_key_version: "encryptionkey@0"
I20260812 16:40:09.534518 29644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:40:09.557634 29644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:40:09.559929 29644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:40:09.560101 29644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:40:09.560355 29644 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:40:09.560688 29644 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:40:09.560752 29644 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:40:09.560796 29644 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:40:09.560823 29644 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:40:09.566946 29644 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.193:39003
I20260812 16:40:09.566998 29915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.193:39003 every 8 connection(s)
I20260812 16:40:09.567335 29644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
Aug 12 16:40:09 dist-test-slave-w4v5 krb5kdc[29431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552809, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:40:09.575654 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 29644
I20260812 16:40:09.575783 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance
I20260812 16:40:09.575989 1015 external_mini_cluster.cc:1428] Setting key 0d5042b53a3e55eac39e5b83aacc42be
I20260812 16:40:09.577098 29609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:09.568179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:40465 (local address 127.0.253.254:39981)
0812 16:40:09.568276 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0812 16:40:09.568279 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:40:09.568458 (+ 179us) server_negotiation.cc:407] Connection header received
0812 16:40:09.569004 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:40:09.569006 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:40:09.569024 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:40:09.569078 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:40:09.570113 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:09.570638 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:09.571709 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:09.571925 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:09.572043 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:09.574149 (+ 2106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:40:09.574151 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:40:09.574151 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:40:09.574163 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:40:09.575527 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:40:09.576031 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:40:09.576033 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:40:09.576034 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:40:09.576129 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:40:09.576427 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:40:09.576428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:40:09.576429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:40:09.576552 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:40:09.576631 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 16:40:09.576957 (+ 326us) server_negotiation.cc:299] Negotiation successful
0812 16:40:09.577036 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 16:40:09.577389 29921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:09.568080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.193:40465)
0812 16:40:09.568314 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:09.568366 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 16:40:09.568885 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:40:09.569164 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:40:09.569176 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:40:09.569792 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:40:09.569986 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:40:09.569991 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:40:09.570741 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:40:09.570742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:40:09.571607 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:40:09.571610 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:40:09.572056 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:40:09.572057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:40:09.572184 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:09.572648 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:40:09.572657 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:40:09.574046 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:40:09.575614 (+ 1568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:40:09.575622 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:40:09.575624 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:40:09.575950 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:40:09.576206 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:40:09.576207 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:40:09.576207 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:40:09.576361 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:40:09.576633 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:40:09.576641 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:40:09.576806 (+ 165us) client_negotiation.cc:769] Sending connection context
0812 16:40:09.576928 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 16:40:09.577117 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":112,"threads_started":1}
I20260812 16:40:09.577963 29917 heartbeater.cc:347] Connected to a master server at 127.0.253.254:39981
I20260812 16:40:09.578069 29917 heartbeater.cc:464] Registering TS with master...
I20260812 16:40:09.578260 29917 heartbeater.cc:511] Master 127.0.253.254:39981 requested a full tablet report, sending...
I20260812 16:40:09.578882 29508 ts_manager.cc:194] Registered new tserver with Master: 0e7940e58fe64aa3a248e733e4e5ccea (127.0.253.193:39003)
I20260812 16:40:09.579866 29508 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.193@KRBTEST.COM'} at 127.0.253.193:40465
WARNING: no policy specified for kudu/127.0.253.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab.
Entry for principal kudu/127.0.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab.
I20260812 16:40:09.602720 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.194:0
--local_ip_for_outbound_sockets=127.0.253.194
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:39981
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:43519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:40:09.686497 29927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:40:09.686717 29927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:40:09.686741 29927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:40:09.687881 29927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:40:09.687919 29927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:40:09.687945 29927 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.194
I20260812 16:40:09.689477 29927 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:43519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:39981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29927
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.194
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:40:09.689672 29927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:40:09.689855 29927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:40:09.694968 29939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:40:09.694962 29947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:40:09.694962 29941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:40:09.695283 29927 server_base.cc:1034] running on GCE node
I20260812 16:40:09.695513 29927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:40:09.695796 29927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:40:09.696965 29927 hybrid_clock.cc:648] HybridClock initialized: now 1786552809696952 us; error 50 us; skew 500 ppm
Aug 12 16:40:09 dist-test-slave-w4v5 krb5kdc[29431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552809, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:40:09.703822 29927 init.cc:377] Logged in from keytab as kudu/127.0.253.194@KRBTEST.COM (short username kudu)
I20260812 16:40:09.704298 29927 webserver.cc:460] Webserver started at http://127.0.253.194:41709/ using document root <none> and password file <none>
I20260812 16:40:09.704541 29927 fs_manager.cc:359] Metadata directory not provided
I20260812 16:40:09.704613 29927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:40:09.704770 29927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:40:09.705870 29927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/data/instance:
uuid: "d585036a34f74e2586fc4f8743561ea6"
format_stamp: "Formatted at 2026-08-12 16:40:09 on dist-test-slave-w4v5"
server_key: "632f75a515053f5d72a5e61f9a2da7bd"
server_key_iv: "3f3c2fe50fa19c104b8674e9b09c1ea8"
server_key_version: "encryptionkey@0"
I20260812 16:40:09.706259 29927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance:
uuid: "d585036a34f74e2586fc4f8743561ea6"
format_stamp: "Formatted at 2026-08-12 16:40:09 on dist-test-slave-w4v5"
server_key: "632f75a515053f5d72a5e61f9a2da7bd"
server_key_iv: "3f3c2fe50fa19c104b8674e9b09c1ea8"
server_key_version: "encryptionkey@0"
I20260812 16:40:09.707783 29927 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:40:09.708585 29964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:40:09.708781 29927 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:40:09.708896 29927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/wal
uuid: "d585036a34f74e2586fc4f8743561ea6"
format_stamp: "Formatted at 2026-08-12 16:40:09 on dist-test-slave-w4v5"
server_key: "632f75a515053f5d72a5e61f9a2da7bd"
server_key_iv: "3f3c2fe50fa19c104b8674e9b09c1ea8"
server_key_version: "encryptionkey@0"
I20260812 16:40:09.708974 29927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:40:09.719673 29927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:40:09.721721 29927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:40:09.721889 29927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:40:09.722133 29927 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:40:09.722448 29927 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:40:09.722503 29927 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:40:09.722558 29927 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:40:09.722589 29927 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:40:09.731868 29927 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.194:44263
I20260812 16:40:09.731920 30209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.194:44263 every 8 connection(s)
I20260812 16:40:09.732255 29927 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
I20260812 16:40:09.738564 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 29927
Aug 12 16:40:09 dist-test-slave-w4v5 krb5kdc[29431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552809, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:40:09.738696 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance
I20260812 16:40:09.738922 1015 external_mini_cluster.cc:1428] Setting key 49055f8f3f2f1577588fcc35b0078d97
I20260812 16:40:09.742403 29609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:09.732946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:33551 (local address 127.0.253.254:39981)
0812 16:40:09.732996 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 16:40:09.732998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:40:09.733202 (+ 204us) server_negotiation.cc:407] Connection header received
0812 16:40:09.733734 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:40:09.733735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:40:09.733745 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:40:09.733801 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:40:09.734806 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:09.735395 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:09.736457 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:09.736651 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:09.736701 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:09.739266 (+ 2565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:40:09.739268 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:40:09.739268 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:40:09.739277 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:40:09.740821 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:40:09.741334 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:40:09.741335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:40:09.741335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:40:09.741389 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:40:09.741690 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:40:09.741691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:40:09.741691 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:40:09.741808 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:40:09.741900 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 16:40:09.742206 (+ 306us) server_negotiation.cc:299] Negotiation successful
0812 16:40:09.742312 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 16:40:09.742621 30216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:09.732900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.194:33551)
0812 16:40:09.733081 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:09.733127 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 16:40:09.733644 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:40:09.733891 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:40:09.733895 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:40:09.734508 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:40:09.734718 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:40:09.734723 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:40:09.735481 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:40:09.735482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:40:09.736353 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:40:09.736356 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:40:09.736724 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:40:09.736725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:40:09.736829 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:09.737381 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:40:09.737391 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:40:09.739150 (+ 1759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:40:09.740907 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:40:09.740915 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:40:09.740917 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:40:09.741257 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:40:09.741456 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:40:09.741457 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:40:09.741457 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:40:09.741620 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:40:09.741895 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:40:09.741904 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:40:09.742058 (+ 154us) client_negotiation.cc:769] Sending connection context
0812 16:40:09.742170 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 16:40:09.742348 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":91,"threads_started":1}
I20260812 16:40:09.743184 30211 heartbeater.cc:347] Connected to a master server at 127.0.253.254:39981
I20260812 16:40:09.743296 30211 heartbeater.cc:464] Registering TS with master...
I20260812 16:40:09.743500 30211 heartbeater.cc:511] Master 127.0.253.254:39981 requested a full tablet report, sending...
I20260812 16:40:09.743924 29508 ts_manager.cc:194] Registered new tserver with Master: d585036a34f74e2586fc4f8743561ea6 (127.0.253.194:44263)
I20260812 16:40:09.744313 29508 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.194@KRBTEST.COM'} at 127.0.253.194:33551
WARNING: no policy specified for kudu/127.0.253.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab.
Entry for principal kudu/127.0.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab.
I20260812 16:40:09.766618 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.195:0
--local_ip_for_outbound_sockets=127.0.253.195
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:39981
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:43519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:40:09.851498 30221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:40:09.851702 30221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:40:09.851727 30221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:40:09.852883 30221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:40:09.852927 30221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:40:09.852953 30221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.195
I20260812 16:40:09.854471 30221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:43519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:39981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.195
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:40:09.854717 30221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:40:09.854966 30221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:40:09.860473 30239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:40:09.860592 30242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:40:09.860658 30238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:40:09.861217 30221 server_base.cc:1034] running on GCE node
I20260812 16:40:09.861500 30221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:40:09.861836 30221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:40:09.862983 30221 hybrid_clock.cc:648] HybridClock initialized: now 1786552809862971 us; error 27 us; skew 500 ppm
Aug 12 16:40:09 dist-test-slave-w4v5 krb5kdc[29431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552809, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:40:09.866531 30221 init.cc:377] Logged in from keytab as kudu/127.0.253.195@KRBTEST.COM (short username kudu)
I20260812 16:40:09.869706 30221 webserver.cc:460] Webserver started at http://127.0.253.195:45275/ using document root <none> and password file <none>
I20260812 16:40:09.870007 30221 fs_manager.cc:359] Metadata directory not provided
I20260812 16:40:09.870091 30221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:40:09.870267 30221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:40:09.871905 30221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/data/instance:
uuid: "d400d8ef8d8c47128e143573073f6aa1"
format_stamp: "Formatted at 2026-08-12 16:40:09 on dist-test-slave-w4v5"
server_key: "90db868f2b1bd673f0d16f3b6c1c461c"
server_key_iv: "f2a504d32ec34742b4894c1bf1554f0c"
server_key_version: "encryptionkey@0"
I20260812 16:40:09.872418 30221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance:
uuid: "d400d8ef8d8c47128e143573073f6aa1"
format_stamp: "Formatted at 2026-08-12 16:40:09 on dist-test-slave-w4v5"
server_key: "90db868f2b1bd673f0d16f3b6c1c461c"
server_key_iv: "f2a504d32ec34742b4894c1bf1554f0c"
server_key_version: "encryptionkey@0"
I20260812 16:40:09.874419 30221 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 16:40:09.875437 30252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:40:09.875628 30221 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:40:09.875736 30221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/wal
uuid: "d400d8ef8d8c47128e143573073f6aa1"
format_stamp: "Formatted at 2026-08-12 16:40:09 on dist-test-slave-w4v5"
server_key: "90db868f2b1bd673f0d16f3b6c1c461c"
server_key_iv: "f2a504d32ec34742b4894c1bf1554f0c"
server_key_version: "encryptionkey@0"
I20260812 16:40:09.875820 30221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:40:09.907760 30221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:40:09.910235 30221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:40:09.910404 30221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:40:09.910660 30221 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:40:09.911006 30221 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:40:09.911065 30221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:40:09.911121 30221 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:40:09.911149 30221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:40:09.918493 30221 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.195:34789
I20260812 16:40:09.918576 30494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.195:34789 every 8 connection(s)
I20260812 16:40:09.918923 30221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
I20260812 16:40:09.924460 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 30221
I20260812 16:40:09.924568 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance
I20260812 16:40:09.924762 1015 external_mini_cluster.cc:1428] Setting key baf1aca50131fc59dafb451146366c36
Aug 12 16:40:09 dist-test-slave-w4v5 krb5kdc[29431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552809, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:40:09.929769 29609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:09.920099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:56211 (local address 127.0.253.254:39981)
0812 16:40:09.920146 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 16:40:09.920149 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:40:09.920165 (+ 16us) server_negotiation.cc:407] Connection header received
0812 16:40:09.921012 (+ 847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:40:09.921014 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:40:09.921024 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:40:09.921070 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:40:09.922521 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:09.923012 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:09.924109 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:09.924369 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:09.924446 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:09.926972 (+ 2526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:40:09.926974 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:40:09.926974 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:40:09.926982 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:40:09.928320 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:40:09.928727 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:40:09.928729 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:40:09.928729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:40:09.928796 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:40:09.929070 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:40:09.929071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:40:09.929071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:40:09.929196 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:40:09.929277 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 16:40:09.929578 (+ 301us) server_negotiation.cc:299] Negotiation successful
0812 16:40:09.929678 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 16:40:09.929999 30501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:09.919820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.195:56211)
0812 16:40:09.920005 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:09.920062 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 16:40:09.920893 (+ 831us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:40:09.921162 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:40:09.921167 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:40:09.922064 (+ 897us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:40:09.922412 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:40:09.922420 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:40:09.923140 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:40:09.923141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:40:09.923992 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:40:09.923995 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:40:09.924455 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:40:09.924456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:40:09.924634 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:09.925254 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:40:09.925269 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:40:09.926868 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:40:09.928406 (+ 1538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:40:09.928413 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:40:09.928416 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:40:09.928646 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:40:09.928870 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:40:09.928871 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:40:09.928871 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:40:09.928993 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:40:09.929267 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:40:09.929274 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:40:09.929442 (+ 168us) client_negotiation.cc:769] Sending connection context
0812 16:40:09.929555 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 16:40:09.929719 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":84,"threads_started":1}
I20260812 16:40:09.930492 30495 heartbeater.cc:347] Connected to a master server at 127.0.253.254:39981
I20260812 16:40:09.930619 30495 heartbeater.cc:464] Registering TS with master...
I20260812 16:40:09.930815 30495 heartbeater.cc:511] Master 127.0.253.254:39981 requested a full tablet report, sending...
I20260812 16:40:09.931277 29508 ts_manager.cc:194] Registered new tserver with Master: d400d8ef8d8c47128e143573073f6aa1 (127.0.253.195:34789)
I20260812 16:40:09.931654 29508 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.195@KRBTEST.COM'} at 127.0.253.195:56211
I20260812 16:40:09.936533 1015 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:40:09.944258 29609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:09.938028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55726 (local address 127.0.253.254:39981)
0812 16:40:09.938090 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 16:40:09.938092 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:40:09.938244 (+ 152us) server_negotiation.cc:407] Connection header received
0812 16:40:09.938318 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:40:09.938319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:40:09.938330 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:40:09.938379 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:40:09.938942 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:09.939502 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:09.940263 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:09.940466 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:09.940542 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:09.941675 (+ 1133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:40:09.941677 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:40:09.941677 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:40:09.941685 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:40:09.942859 (+ 1174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:40:09.943407 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:40:09.943409 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:40:09.943409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:40:09.943470 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:40:09.943671 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:40:09.943672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:40:09.943673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:40:09.943830 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:40:09.943939 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 16:40:09.944070 (+ 131us) server_negotiation.cc:299] Negotiation successful
0812 16:40:09.944147 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 16:40:09.945672 29508 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55726:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:40:09.947254 29508 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:40:09.954121 30526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:09.949608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:44263 (local address 127.0.0.1:44476)
0812 16:40:09.949969 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:09.949981 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:40:09.950037 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:40:09.950374 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:40:09.950375 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:40:09.950376 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:40:09.950421 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:40:09.950422 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:40:09.951573 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:40:09.951574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:40:09.952915 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:40:09.952917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:40:09.953835 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:40:09.953836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:40:09.953893 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:09.953903 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 16:40:09.953993 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 16:40:09.954047 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"mutex_wait_us":23,"thread_start_us":69,"threads_started":1}
I20260812 16:40:09.954147 30528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:09.950016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44476 (local address 127.0.253.194:44263)
0812 16:40:09.950173 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 16:40:09.950179 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:40:09.950190 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:40:09.950240 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:40:09.950242 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:40:09.950261 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:40:09.950515 (+ 254us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:40:09.950600 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:09.951459 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:09.953009 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:09.953755 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:09.953836 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:09.953895 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 16:40:09.954017 (+ 122us) server_negotiation.cc:299] Negotiation successful
0812 16:40:09.954067 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":86,"threads_started":1}
I20260812 16:40:09.954567 30519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:09.949852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.195:34789 (local address 127.0.0.1:39752)
0812 16:40:09.950277 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:09.950289 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:40:09.950331 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:40:09.950699 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:40:09.950700 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:40:09.950702 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:40:09.950753 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:40:09.950755 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:40:09.952013 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:40:09.952014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:40:09.952926 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:40:09.952928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:40:09.954347 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:40:09.954349 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:40:09.954407 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:09.954411 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:40:09.954464 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 16:40:09.954504 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":73,"threads_started":1}
I20260812 16:40:09.954846 30529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:09.950300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39752 (local address 127.0.253.195:34789)
0812 16:40:09.950531 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0812 16:40:09.950536 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:40:09.950547 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:40:09.950586 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:40:09.950588 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:40:09.950604 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:40:09.950687 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:40:09.950819 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:09.951901 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:09.953043 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:09.953756 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:09.953836 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:09.953894 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 16:40:09.954706 (+ 812us) server_negotiation.cc:299] Negotiation successful
0812 16:40:09.954764 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":103,"threads_started":1}
I20260812 16:40:09.954989 30520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:09.949602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:39003 (local address 127.0.0.1:49130)
0812 16:40:09.949749 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:09.949795 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 16:40:09.949884 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:40:09.950096 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:40:09.950099 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:40:09.950101 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:40:09.950179 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:40:09.950181 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:40:09.951824 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:40:09.951825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:40:09.952911 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:40:09.952914 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:40:09.954731 (+ 1817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:40:09.954732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:40:09.954810 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:09.954814 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:40:09.954845 (+ 31us) client_negotiation.cc:240] Negotiation successful
0812 16:40:09.954891 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":3,"thread_start_us":97,"threads_started":1}
I20260812 16:40:09.954931 30073 tablet_service.cc:1467] Processing CreateTablet for tablet 15a56323fe38453486c84ae761078bb2 (DEFAULT_TABLE table=test-table [id=eb6d57d348754ea5be240110e4518749]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:40:09.955369 30073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15a56323fe38453486c84ae761078bb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:40:09.955406 30523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:09.949721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49130 (local address 127.0.253.193:39003)
0812 16:40:09.949918 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 16:40:09.949924 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:40:09.949937 (+ 13us) server_negotiation.cc:407] Connection header received
0812 16:40:09.949978 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:40:09.949980 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:40:09.949995 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:40:09.950092 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:40:09.950504 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:09.951715 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:09.953934 (+ 2219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:09.954627 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:09.955183 (+ 556us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:09.955267 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 16:40:09.955294 (+ 27us) server_negotiation.cc:299] Negotiation successful
0812 16:40:09.955340 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":90,"threads_started":1}
I20260812 16:40:09.955473 30353 tablet_service.cc:1467] Processing CreateTablet for tablet 15a56323fe38453486c84ae761078bb2 (DEFAULT_TABLE table=test-table [id=eb6d57d348754ea5be240110e4518749]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:40:09.955799 30353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15a56323fe38453486c84ae761078bb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:40:09.956012 29785 tablet_service.cc:1467] Processing CreateTablet for tablet 15a56323fe38453486c84ae761078bb2 (DEFAULT_TABLE table=test-table [id=eb6d57d348754ea5be240110e4518749]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:40:09.956334 29785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15a56323fe38453486c84ae761078bb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:40:09.959012 30531 tablet_bootstrap.cc:492] T 15a56323fe38453486c84ae761078bb2 P d400d8ef8d8c47128e143573073f6aa1: Bootstrap starting.
I20260812 16:40:09.959358 30532 tablet_bootstrap.cc:492] T 15a56323fe38453486c84ae761078bb2 P 0e7940e58fe64aa3a248e733e4e5ccea: Bootstrap starting.
I20260812 16:40:09.959836 30531 tablet_bootstrap.cc:654] T 15a56323fe38453486c84ae761078bb2 P d400d8ef8d8c47128e143573073f6aa1: Neither blocks nor log segments found. Creating new log.
I20260812 16:40:09.960041 30532 tablet_bootstrap.cc:654] T 15a56323fe38453486c84ae761078bb2 P 0e7940e58fe64aa3a248e733e4e5ccea: Neither blocks nor log segments found. Creating new log.
I20260812 16:40:09.960562 30531 log.cc:824] T 15a56323fe38453486c84ae761078bb2 P d400d8ef8d8c47128e143573073f6aa1: Log is configured to *not* fsync() on all Append() calls
I20260812 16:40:09.960671 30532 log.cc:824] T 15a56323fe38453486c84ae761078bb2 P 0e7940e58fe64aa3a248e733e4e5ccea: Log is configured to *not* fsync() on all Append() calls
I20260812 16:40:09.961386 30532 tablet_bootstrap.cc:492] T 15a56323fe38453486c84ae761078bb2 P 0e7940e58fe64aa3a248e733e4e5ccea: No bootstrap required, opened a new log
I20260812 16:40:09.961426 30531 tablet_bootstrap.cc:492] T 15a56323fe38453486c84ae761078bb2 P d400d8ef8d8c47128e143573073f6aa1: No bootstrap required, opened a new log
I20260812 16:40:09.961477 30532 ts_tablet_manager.cc:1397] T 15a56323fe38453486c84ae761078bb2 P 0e7940e58fe64aa3a248e733e4e5ccea: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:40:09.961504 30531 ts_tablet_manager.cc:1397] T 15a56323fe38453486c84ae761078bb2 P d400d8ef8d8c47128e143573073f6aa1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 16:40:09.962836 30532 raft_consensus.cc:348] T 15a56323fe38453486c84ae761078bb2 P 0e7940e58fe64aa3a248e733e4e5ccea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7940e58fe64aa3a248e733e4e5ccea" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 39003 } } peers { permanent_uuid: "d585036a34f74e2586fc4f8743561ea6" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44263 } } peers { permanent_uuid: "d400d8ef8d8c47128e143573073f6aa1" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 34789 } }
I20260812 16:40:09.962836 30531 raft_consensus.cc:348] T 15a56323fe38453486c84ae761078bb2 P d400d8ef8d8c47128e143573073f6aa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7940e58fe64aa3a248e733e4e5ccea" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 39003 } } peers { permanent_uuid: "d585036a34f74e2586fc4f8743561ea6" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44263 } } peers { permanent_uuid: "d400d8ef8d8c47128e143573073f6aa1" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 34789 } }
I20260812 16:40:09.962997 30531 raft_consensus.cc:374] T 15a56323fe38453486c84ae761078bb2 P d400d8ef8d8c47128e143573073f6aa1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:40:09.963001 30532 raft_consensus.cc:374] T 15a56323fe38453486c84ae761078bb2 P 0e7940e58fe64aa3a248e733e4e5ccea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:40:09.963019 30531 raft_consensus.cc:729] T 15a56323fe38453486c84ae761078bb2 P d400d8ef8d8c47128e143573073f6aa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d400d8ef8d8c47128e143573073f6aa1, State: Initialized, Role: FOLLOWER
I20260812 16:40:09.963050 30532 raft_consensus.cc:729] T 15a56323fe38453486c84ae761078bb2 P 0e7940e58fe64aa3a248e733e4e5ccea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e7940e58fe64aa3a248e733e4e5ccea, State: Initialized, Role: FOLLOWER
I20260812 16:40:09.963193 30531 consensus_queue.cc:260] T 15a56323fe38453486c84ae761078bb2 P d400d8ef8d8c47128e143573073f6aa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7940e58fe64aa3a248e733e4e5ccea" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 39003 } } peers { permanent_uuid: "d585036a34f74e2586fc4f8743561ea6" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44263 } } peers { permanent_uuid: "d400d8ef8d8c47128e143573073f6aa1" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 34789 } }
I20260812 16:40:09.963255 30532 consensus_queue.cc:260] T 15a56323fe38453486c84ae761078bb2 P 0e7940e58fe64aa3a248e733e4e5ccea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7940e58fe64aa3a248e733e4e5ccea" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 39003 } } peers { permanent_uuid: "d585036a34f74e2586fc4f8743561ea6" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44263 } } peers { permanent_uuid: "d400d8ef8d8c47128e143573073f6aa1" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 34789 } }
I20260812 16:40:09.963522 30495 heartbeater.cc:503] Master 127.0.253.254:39981 was elected leader, sending a full tablet report...
I20260812 16:40:09.963578 30532 ts_tablet_manager.cc:1428] T 15a56323fe38453486c84ae761078bb2 P 0e7940e58fe64aa3a248e733e4e5ccea: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:40:09.963594 29917 heartbeater.cc:503] Master 127.0.253.254:39981 was elected leader, sending a full tablet report...
I20260812 16:40:09.963775 30531 ts_tablet_manager.cc:1428] T 15a56323fe38453486c84ae761078bb2 P d400d8ef8d8c47128e143573073f6aa1: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 16:40:09.963866 30530 tablet_bootstrap.cc:492] T 15a56323fe38453486c84ae761078bb2 P d585036a34f74e2586fc4f8743561ea6: Bootstrap starting.
I20260812 16:40:09.964911 30530 tablet_bootstrap.cc:654] T 15a56323fe38453486c84ae761078bb2 P d585036a34f74e2586fc4f8743561ea6: Neither blocks nor log segments found. Creating new log.
I20260812 16:40:09.965474 30530 log.cc:824] T 15a56323fe38453486c84ae761078bb2 P d585036a34f74e2586fc4f8743561ea6: Log is configured to *not* fsync() on all Append() calls
I20260812 16:40:09.966192 30530 tablet_bootstrap.cc:492] T 15a56323fe38453486c84ae761078bb2 P d585036a34f74e2586fc4f8743561ea6: No bootstrap required, opened a new log
I20260812 16:40:09.966245 30530 ts_tablet_manager.cc:1397] T 15a56323fe38453486c84ae761078bb2 P d585036a34f74e2586fc4f8743561ea6: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260812 16:40:09.967698 30530 raft_consensus.cc:348] T 15a56323fe38453486c84ae761078bb2 P d585036a34f74e2586fc4f8743561ea6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7940e58fe64aa3a248e733e4e5ccea" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 39003 } } peers { permanent_uuid: "d585036a34f74e2586fc4f8743561ea6" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44263 } } peers { permanent_uuid: "d400d8ef8d8c47128e143573073f6aa1" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 34789 } }
I20260812 16:40:09.967840 30530 raft_consensus.cc:374] T 15a56323fe38453486c84ae761078bb2 P d585036a34f74e2586fc4f8743561ea6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:40:09.967862 30530 raft_consensus.cc:729] T 15a56323fe38453486c84ae761078bb2 P d585036a34f74e2586fc4f8743561ea6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d585036a34f74e2586fc4f8743561ea6, State: Initialized, Role: FOLLOWER
I20260812 16:40:09.967943 30530 consensus_queue.cc:260] T 15a56323fe38453486c84ae761078bb2 P d585036a34f74e2586fc4f8743561ea6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7940e58fe64aa3a248e733e4e5ccea" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 39003 } } peers { permanent_uuid: "d585036a34f74e2586fc4f8743561ea6" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44263 } } peers { permanent_uuid: "d400d8ef8d8c47128e143573073f6aa1" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 34789 } }
I20260812 16:40:09.968147 30530 ts_tablet_manager.cc:1428] T 15a56323fe38453486c84ae761078bb2 P d585036a34f74e2586fc4f8743561ea6: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 16:40:09.968217 30211 heartbeater.cc:503] Master 127.0.253.254:39981 was elected leader, sending a full tablet report...
W20260812 16:40:09.982785 30214 tablet.cc:2369] T 15a56323fe38453486c84ae761078bb2 P d585036a34f74e2586fc4f8743561ea6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 16:40:10.068069 29919 tablet.cc:2369] T 15a56323fe38453486c84ae761078bb2 P 0e7940e58fe64aa3a248e733e4e5ccea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:40:10.091473 30542 raft_consensus.cc:482] T 15a56323fe38453486c84ae761078bb2 P d585036a34f74e2586fc4f8743561ea6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:40:10.091614 30542 raft_consensus.cc:504] T 15a56323fe38453486c84ae761078bb2 P d585036a34f74e2586fc4f8743561ea6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7940e58fe64aa3a248e733e4e5ccea" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 39003 } } peers { permanent_uuid: "d585036a34f74e2586fc4f8743561ea6" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44263 } } peers { permanent_uuid: "d400d8ef8d8c47128e143573073f6aa1" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 34789 } }
I20260812 16:40:10.092105 30542 leader_election.cc:288] T 15a56323fe38453486c84ae761078bb2 P d585036a34f74e2586fc4f8743561ea6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e7940e58fe64aa3a248e733e4e5ccea (127.0.253.193:39003), d400d8ef8d8c47128e143573073f6aa1 (127.0.253.195:34789)
I20260812 16:40:10.095281 30529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:10.092509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:51689 (local address 127.0.253.195:34789)
0812 16:40:10.092639 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 16:40:10.092641 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:40:10.092651 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:40:10.092703 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:40:10.092704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:40:10.092715 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:40:10.092788 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:40:10.092949 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:10.093407 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:10.094470 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:10.094858 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:10.094946 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:10.094956 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 16:40:10.095144 (+ 188us) server_negotiation.cc:299] Negotiation successful
0812 16:40:10.095183 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260812 16:40:10.095279 30523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:10.092419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:37825 (local address 127.0.253.193:39003)
0812 16:40:10.092473 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 16:40:10.092475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:40:10.092490 (+ 15us) server_negotiation.cc:407] Connection header received
0812 16:40:10.092572 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:40:10.092573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:40:10.092585 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:40:10.092654 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:40:10.092829 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:10.093323 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:10.094482 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:10.094859 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:10.094947 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:10.094956 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 16:40:10.095164 (+ 208us) server_negotiation.cc:299] Negotiation successful
0812 16:40:10.095205 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 16:40:10.095302 30216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:10.092346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:39003 (local address 127.0.253.194:37825)
0812 16:40:10.092432 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:10.092446 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 16:40:10.092497 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:40:10.092659 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:40:10.092660 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:40:10.092662 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:40:10.092736 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:40:10.092738 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:40:10.093412 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:40:10.093413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:40:10.094401 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:40:10.094404 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:40:10.094947 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:40:10.094948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:40:10.095037 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:10.095046 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 16:40:10.095151 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 16:40:10.095196 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63,"mutex_wait_us":1}
I20260812 16:40:10.095551 30543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:10.092352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.195:34789 (local address 127.0.253.194:51689)
0812 16:40:10.092600 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:10.092609 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 16:40:10.092651 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:40:10.092792 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:40:10.092792 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:40:10.092794 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:40:10.092877 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:40:10.092879 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:40:10.093490 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:40:10.093491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:40:10.094399 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:40:10.094403 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:40:10.094946 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:40:10.094947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:40:10.095021 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:10.095025 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:40:10.095138 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 16:40:10.095183 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"mutex_wait_us":73,"thread_start_us":122,"threads_started":1}
I20260812 16:40:10.095691 29815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15a56323fe38453486c84ae761078bb2" candidate_uuid: "d585036a34f74e2586fc4f8743561ea6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e7940e58fe64aa3a248e733e4e5ccea" is_pre_election: true
I20260812 16:40:10.095851 29815 raft_consensus.cc:2454] T 15a56323fe38453486c84ae761078bb2 P 0e7940e58fe64aa3a248e733e4e5ccea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d585036a34f74e2586fc4f8743561ea6 in term 0.
I20260812 16:40:10.095901 30399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15a56323fe38453486c84ae761078bb2" candidate_uuid: "d585036a34f74e2586fc4f8743561ea6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d400d8ef8d8c47128e143573073f6aa1" is_pre_election: true
I20260812 16:40:10.096050 30399 raft_consensus.cc:2454] T 15a56323fe38453486c84ae761078bb2 P d400d8ef8d8c47128e143573073f6aa1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d585036a34f74e2586fc4f8743561ea6 in term 0.
I20260812 16:40:10.096179 29966 leader_election.cc:302] T 15a56323fe38453486c84ae761078bb2 P d585036a34f74e2586fc4f8743561ea6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e7940e58fe64aa3a248e733e4e5ccea, d585036a34f74e2586fc4f8743561ea6; no voters:
I20260812 16:40:10.096313 30542 raft_consensus.cc:2781] T 15a56323fe38453486c84ae761078bb2 P d585036a34f74e2586fc4f8743561ea6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:40:10.096371 30542 raft_consensus.cc:482] T 15a56323fe38453486c84ae761078bb2 P d585036a34f74e2586fc4f8743561ea6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:40:10.096395 30542 raft_consensus.cc:3037] T 15a56323fe38453486c84ae761078bb2 P d585036a34f74e2586fc4f8743561ea6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:40:10.097142 30542 raft_consensus.cc:504] T 15a56323fe38453486c84ae761078bb2 P d585036a34f74e2586fc4f8743561ea6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7940e58fe64aa3a248e733e4e5ccea" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 39003 } } peers { permanent_uuid: "d585036a34f74e2586fc4f8743561ea6" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44263 } } peers { permanent_uuid: "d400d8ef8d8c47128e143573073f6aa1" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 34789 } }
I20260812 16:40:10.097292 30542 leader_election.cc:288] T 15a56323fe38453486c84ae761078bb2 P d585036a34f74e2586fc4f8743561ea6 [CANDIDATE]: Term 1 election: Requested vote from peers 0e7940e58fe64aa3a248e733e4e5ccea (127.0.253.193:39003), d400d8ef8d8c47128e143573073f6aa1 (127.0.253.195:34789)
I20260812 16:40:10.097474 29815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15a56323fe38453486c84ae761078bb2" candidate_uuid: "d585036a34f74e2586fc4f8743561ea6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e7940e58fe64aa3a248e733e4e5ccea"
I20260812 16:40:10.097569 29815 raft_consensus.cc:3037] T 15a56323fe38453486c84ae761078bb2 P 0e7940e58fe64aa3a248e733e4e5ccea [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:40:10.097504 30399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15a56323fe38453486c84ae761078bb2" candidate_uuid: "d585036a34f74e2586fc4f8743561ea6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d400d8ef8d8c47128e143573073f6aa1"
I20260812 16:40:10.097667 30399 raft_consensus.cc:3037] T 15a56323fe38453486c84ae761078bb2 P d400d8ef8d8c47128e143573073f6aa1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:40:10.098292 29815 raft_consensus.cc:2454] T 15a56323fe38453486c84ae761078bb2 P 0e7940e58fe64aa3a248e733e4e5ccea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d585036a34f74e2586fc4f8743561ea6 in term 1.
I20260812 16:40:10.098474 30399 raft_consensus.cc:2454] T 15a56323fe38453486c84ae761078bb2 P d400d8ef8d8c47128e143573073f6aa1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d585036a34f74e2586fc4f8743561ea6 in term 1.
I20260812 16:40:10.098479 29966 leader_election.cc:302] T 15a56323fe38453486c84ae761078bb2 P d585036a34f74e2586fc4f8743561ea6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e7940e58fe64aa3a248e733e4e5ccea, d585036a34f74e2586fc4f8743561ea6; no voters:
I20260812 16:40:10.098670 30542 raft_consensus.cc:2781] T 15a56323fe38453486c84ae761078bb2 P d585036a34f74e2586fc4f8743561ea6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:40:10.098883 30542 raft_consensus.cc:686] T 15a56323fe38453486c84ae761078bb2 P d585036a34f74e2586fc4f8743561ea6 [term 1 LEADER]: Becoming Leader. State: Replica: d585036a34f74e2586fc4f8743561ea6, State: Running, Role: LEADER
I20260812 16:40:10.098985 30542 consensus_queue.cc:237] T 15a56323fe38453486c84ae761078bb2 P d585036a34f74e2586fc4f8743561ea6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7940e58fe64aa3a248e733e4e5ccea" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 39003 } } peers { permanent_uuid: "d585036a34f74e2586fc4f8743561ea6" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44263 } } peers { permanent_uuid: "d400d8ef8d8c47128e143573073f6aa1" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 34789 } }
I20260812 16:40:10.099874 29508 catalog_manager.cc:5495] T 15a56323fe38453486c84ae761078bb2 P d585036a34f74e2586fc4f8743561ea6 reported cstate change: term changed from 0 to 1, leader changed from <none> to d585036a34f74e2586fc4f8743561ea6 (127.0.253.194). New cstate: current_term: 1 leader_uuid: "d585036a34f74e2586fc4f8743561ea6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7940e58fe64aa3a248e733e4e5ccea" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d585036a34f74e2586fc4f8743561ea6" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d400d8ef8d8c47128e143573073f6aa1" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 34789 } health_report { overall_health: UNKNOWN } } }
I20260812 16:40:10.135058 30528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:10.132524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44478 (local address 127.0.253.194:44263)
0812 16:40:10.132595 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 16:40:10.132597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:40:10.132607 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:40:10.132689 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:40:10.132694 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:40:10.132716 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:40:10.132821 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:40:10.132972 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:10.133460 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:10.134231 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:10.134434 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:10.134514 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:10.134678 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:40:10.134798 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:40:10.134899 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 16:40:10.134918 (+ 19us) server_negotiation.cc:299] Negotiation successful
0812 16:40:10.134940 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 16:40:10.137099 30399 raft_consensus.cc:1261] T 15a56323fe38453486c84ae761078bb2 P d400d8ef8d8c47128e143573073f6aa1 [term 1 FOLLOWER]: Refusing update from remote peer d585036a34f74e2586fc4f8743561ea6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:40:10.137140 29815 raft_consensus.cc:1261] T 15a56323fe38453486c84ae761078bb2 P 0e7940e58fe64aa3a248e733e4e5ccea [term 1 FOLLOWER]: Refusing update from remote peer d585036a34f74e2586fc4f8743561ea6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:40:10.137527 30542 consensus_queue.cc:1035] T 15a56323fe38453486c84ae761078bb2 P d585036a34f74e2586fc4f8743561ea6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d400d8ef8d8c47128e143573073f6aa1" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 34789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:40:10.137710 30544 consensus_queue.cc:1035] T 15a56323fe38453486c84ae761078bb2 P d585036a34f74e2586fc4f8743561ea6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e7940e58fe64aa3a248e733e4e5ccea" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:40:10.142398 30550 mvcc.cc:205] Tried to move back new op lower bound from 7317720310318551040 to 7317720310166171648. Current Snapshot: MvccSnapshot[applied={T|T < 7317720310318551040 or (T in {7317720310318551040})}]
I20260812 16:40:10.142632 29508 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:40:10.142751 29508 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:40:10.143587 29508 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea: Sending DeleteTablet for 3 replicas of tablet 15a56323fe38453486c84ae761078bb2
I20260812 16:40:10.143949 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 29644
I20260812 16:40:10.143953 30073 tablet_service.cc:1514] Processing DeleteTablet for tablet 15a56323fe38453486c84ae761078bb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:40:10 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:44476
I20260812 16:40:10.143980 29785 tablet_service.cc:1514] Processing DeleteTablet for tablet 15a56323fe38453486c84ae761078bb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:40:10 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:49130
I20260812 16:40:10.144479 30353 tablet_service.cc:1514] Processing DeleteTablet for tablet 15a56323fe38453486c84ae761078bb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:40:10 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:39752
I20260812 16:40:10.153872 30563 tablet_replica.cc:330] T 15a56323fe38453486c84ae761078bb2 P d400d8ef8d8c47128e143573073f6aa1: stopping tablet replica
I20260812 16:40:10.154036 30563 raft_consensus.cc:2229] T 15a56323fe38453486c84ae761078bb2 P d400d8ef8d8c47128e143573073f6aa1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:40:10.154196 30563 raft_consensus.cc:2258] T 15a56323fe38453486c84ae761078bb2 P d400d8ef8d8c47128e143573073f6aa1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:40:10.154408 30562 tablet_replica.cc:330] T 15a56323fe38453486c84ae761078bb2 P d585036a34f74e2586fc4f8743561ea6: stopping tablet replica
I20260812 16:40:10.154511 30562 raft_consensus.cc:2229] T 15a56323fe38453486c84ae761078bb2 P d585036a34f74e2586fc4f8743561ea6 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:40:10.154603 30562 raft_consensus.cc:2258] T 15a56323fe38453486c84ae761078bb2 P d585036a34f74e2586fc4f8743561ea6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:40:10.155339 30563 ts_tablet_manager.cc:1905] T 15a56323fe38453486c84ae761078bb2 P d400d8ef8d8c47128e143573073f6aa1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:40:10.156251 30562 ts_tablet_manager.cc:1905] T 15a56323fe38453486c84ae761078bb2 P d585036a34f74e2586fc4f8743561ea6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:40:10.157284 30563 ts_tablet_manager.cc:1918] T 15a56323fe38453486c84ae761078bb2 P d400d8ef8d8c47128e143573073f6aa1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:40:10.157434 30563 log.cc:1196] T 15a56323fe38453486c84ae761078bb2 P d400d8ef8d8c47128e143573073f6aa1: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/wal/wals/15a56323fe38453486c84ae761078bb2
I20260812 16:40:10.157816 30563 ts_tablet_manager.cc:1939] T 15a56323fe38453486c84ae761078bb2 P d400d8ef8d8c47128e143573073f6aa1: Deleting consensus metadata
I20260812 16:40:10.158054 30562 ts_tablet_manager.cc:1918] T 15a56323fe38453486c84ae761078bb2 P d585036a34f74e2586fc4f8743561ea6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:40:10.158134 30562 log.cc:1196] T 15a56323fe38453486c84ae761078bb2 P d585036a34f74e2586fc4f8743561ea6: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/wal/wals/15a56323fe38453486c84ae761078bb2
I20260812 16:40:10.158424 30562 ts_tablet_manager.cc:1939] T 15a56323fe38453486c84ae761078bb2 P d585036a34f74e2586fc4f8743561ea6: Deleting consensus metadata
I20260812 16:40:10.158529 29488 catalog_manager.cc:4841] TS d400d8ef8d8c47128e143573073f6aa1 (127.0.253.195:34789): tablet 15a56323fe38453486c84ae761078bb2 (table test-table [id=eb6d57d348754ea5be240110e4518749]) successfully deleted
I20260812 16:40:10.158818 29488 catalog_manager.cc:4841] TS d585036a34f74e2586fc4f8743561ea6 (127.0.253.194:44263): tablet 15a56323fe38453486c84ae761078bb2 (table test-table [id=eb6d57d348754ea5be240110e4518749]) successfully deleted
W20260812 16:40:10.173350 29485 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.253.193:39003 (error 108)
I20260812 16:40:10.173616 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 29927
I20260812 16:40:10.180878 30519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:10.180746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:39003 (local address 127.0.0.1:49134)
0812 16:40:10.180799 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:10.180834 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.193:39003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 16:40:10.181007 29485 catalog_manager.cc:4584] TS 0e7940e58fe64aa3a248e733e4e5ccea (127.0.253.193:39003): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 15a56323fe38453486c84ae761078bb2: Network error: Client connection negotiation failed: client connection to 127.0.253.193:39003: connect: Connection refused (error 111)
I20260812 16:40:10.189613 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 30221
I20260812 16:40:10.205448 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 29457
I20260812 16:40:30.219872 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:39981
--webserver_interface=127.0.253.254
--webserver_port=33263
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:43519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:39981
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:40:30.307853 30567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:40:30.308064 30567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:40:30.308086 30567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:40:30.309288 30567 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 16:40:30.309332 30567 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 16:40:30.309346 30567 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 16:40:30.309357 30567 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:40:30.309370 30567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:40:30.309383 30567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:40:30.309425 30567 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:40:30.309455 30567 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:40:30.310987 30567 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:43519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:39981
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:39981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=33263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:40:30.311295 30567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:40:30.311565 30567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:40:30.321565 30590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:40:30.321565 30588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:40:30.321568 30585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:40:30.321909 30567 server_base.cc:1034] running on GCE node
I20260812 16:40:30.322224 30567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:40:30.322549 30567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:40:30.323724 30567 hybrid_clock.cc:648] HybridClock initialized: now 1786552830323708 us; error 41 us; skew 500 ppm
Aug 12 16:40:30 dist-test-slave-w4v5 krb5kdc[29431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552830, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:40:30.326421 30567 init.cc:377] Logged in from keytab as kudu/127.0.253.254@KRBTEST.COM (short username kudu)
I20260812 16:40:30.326923 30567 webserver.cc:460] Webserver started at http://127.0.253.254:33263/ using document root <none> and password file <none>
I20260812 16:40:30.327252 30567 fs_manager.cc:359] Metadata directory not provided
I20260812 16:40:30.327389 30567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:40:30.329181 30567 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:40:30.329829 30604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:40:30.330004 30567 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:40:30.330109 30567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "3bddd07e41ca4d92aaf62a0b496011ea"
format_stamp: "Formatted at 2026-08-12 16:40:09 on dist-test-slave-w4v5"
server_key: "ba0b99e8997ca623764cf5d495c46212"
server_key_iv: "ae6fc5547a948212895046dc3e8bed98"
server_key_version: "encryptionkey@0"
I20260812 16:40:30.330562 30567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:40:30.343683 30567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:40:30.345710 30567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:40:30.345865 30567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:40:30.350299 30567 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:39981
I20260812 16:40:30.350335 30703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:39981 every 8 connection(s)
I20260812 16:40:30.350764 30567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:40:30.352516 30704 sys_catalog.cc:263] Verifying existing consensus state
I20260812 16:40:30.353284 30704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea: Bootstrap starting.
I20260812 16:40:30.356781 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 30567
I20260812 16:40:30.357085 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.193:39003
--local_ip_for_outbound_sockets=127.0.253.193
--tserver_master_addrs=127.0.253.254:39981
--webserver_port=46433
--webserver_interface=127.0.253.193
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:43519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:40:30.364152 30704 log.cc:824] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea: Log is configured to *not* fsync() on all Append() calls
I20260812 16:40:30.367362 30704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 16:40:30.367787 30704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea: Bootstrap complete.
I20260812 16:40:30.372605 30704 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bddd07e41ca4d92aaf62a0b496011ea" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39981 } }
I20260812 16:40:30.372989 30704 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bddd07e41ca4d92aaf62a0b496011ea, State: Initialized, Role: FOLLOWER
I20260812 16:40:30.373173 30704 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bddd07e41ca4d92aaf62a0b496011ea" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39981 } }
I20260812 16:40:30.373247 30704 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:40:30.373311 30704 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:40:30.373364 30704 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [term 1 FOLLOWER]: Advancing to term 2
I20260812 16:40:30.374893 30704 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bddd07e41ca4d92aaf62a0b496011ea" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39981 } }
I20260812 16:40:30.375021 30704 leader_election.cc:302] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3bddd07e41ca4d92aaf62a0b496011ea; no voters:
I20260812 16:40:30.375234 30704 leader_election.cc:288] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 16:40:30.375346 30710 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [term 2 FOLLOWER]: Leader election won for term 2
I20260812 16:40:30.375519 30704 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:40:30.375626 30710 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [term 2 LEADER]: Becoming Leader. State: Replica: 3bddd07e41ca4d92aaf62a0b496011ea, State: Running, Role: LEADER
I20260812 16:40:30.375795 30710 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bddd07e41ca4d92aaf62a0b496011ea" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39981 } }
I20260812 16:40:30.376180 30711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3bddd07e41ca4d92aaf62a0b496011ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bddd07e41ca4d92aaf62a0b496011ea" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39981 } } }
I20260812 16:40:30.376284 30711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [sys.catalog]: This master's current role is: LEADER
I20260812 16:40:30.376497 30713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3bddd07e41ca4d92aaf62a0b496011ea. Latest consensus state: current_term: 2 leader_uuid: "3bddd07e41ca4d92aaf62a0b496011ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bddd07e41ca4d92aaf62a0b496011ea" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39981 } } }
I20260812 16:40:30.376561 30713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [sys.catalog]: This master's current role is: LEADER
I20260812 16:40:30.376945 30728 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:40:30.378051 30728 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:40:30.378163 30728 catalog_manager.cc:1259] Loaded cluster ID: 7580f6e5149941e8865a4353b78d6187
I20260812 16:40:30.378201 30728 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:40:30.379633 30728 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:40:30.379829 30728 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea: Loaded TSK: 0
I20260812 16:40:30.391547 30728 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea: Generated new TSK 1
I20260812 16:40:30.391681 30728 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 16:40:30.456038 30707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:40:30.456279 30707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:40:30.456319 30707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:40:30.457451 30707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:40:30.457516 30707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:40:30.457563 30707 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.193
I20260812 16:40:30.459136 30707 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:43519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.193:39003
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.253.193
--webserver_port=46433
--tserver_master_addrs=127.0.253.254:39981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30707
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.193
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:40:30.459396 30707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:40:30.459661 30707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:40:30.465014 30759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:40:30.465032 30765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:40:30.465165 30762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:40:30.465420 30707 server_base.cc:1034] running on GCE node
I20260812 16:40:30.465634 30707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:40:30.465891 30707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:40:30.467056 30707 hybrid_clock.cc:648] HybridClock initialized: now 1786552830467030 us; error 38 us; skew 500 ppm
Aug 12 16:40:30 dist-test-slave-w4v5 krb5kdc[29431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552830, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:40:30.469933 30707 init.cc:377] Logged in from keytab as kudu/127.0.253.193@KRBTEST.COM (short username kudu)
I20260812 16:40:30.470504 30707 webserver.cc:460] Webserver started at http://127.0.253.193:46433/ using document root <none> and password file <none>
I20260812 16:40:30.470746 30707 fs_manager.cc:359] Metadata directory not provided
I20260812 16:40:30.470830 30707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:40:30.472354 30707 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:40:30.473143 30780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:40:30.473343 30707 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:40:30.473446 30707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/wal
uuid: "0e7940e58fe64aa3a248e733e4e5ccea"
format_stamp: "Formatted at 2026-08-12 16:40:09 on dist-test-slave-w4v5"
server_key: "277a689f10147fc0e9b471a980e66894"
server_key_iv: "d58bb779abd4e0042bb7dca4fff0fa46"
server_key_version: "encryptionkey@0"
I20260812 16:40:30.473810 30707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:40:30.493070 30707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:40:30.495818 30707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:40:30.495990 30707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:40:30.496258 30707 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:40:30.496775 30790 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 16:40:30.498279 30707 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 16:40:30.498332 30707 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 16:40:30.498366 30707 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 16:40:30.499068 30707 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 16:40:30.499173 30707 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.001s
I20260812 16:40:30.499203 30790 tablet_bootstrap.cc:492] T 15a56323fe38453486c84ae761078bb2 P 0e7940e58fe64aa3a248e733e4e5ccea: Bootstrap starting.
I20260812 16:40:30.507150 30707 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.193:39003
I20260812 16:40:30.507221 30985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.193:39003 every 8 connection(s)
I20260812 16:40:30.507869 30707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
I20260812 16:40:30.511479 30790 log.cc:824] T 15a56323fe38453486c84ae761078bb2 P 0e7940e58fe64aa3a248e733e4e5ccea: Log is configured to *not* fsync() on all Append() calls
I20260812 16:40:30.514122 30790 tablet_bootstrap.cc:492] T 15a56323fe38453486c84ae761078bb2 P 0e7940e58fe64aa3a248e733e4e5ccea: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 16:40:30.514442 30790 tablet_bootstrap.cc:492] T 15a56323fe38453486c84ae761078bb2 P 0e7940e58fe64aa3a248e733e4e5ccea: Bootstrap complete.
I20260812 16:40:30.515091 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 30707
I20260812 16:40:30.515400 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.194:44263
--local_ip_for_outbound_sockets=127.0.253.194
--tserver_master_addrs=127.0.253.254:39981
--webserver_port=41709
--webserver_interface=127.0.253.194
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:43519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:40:30.515705 30790 ts_tablet_manager.cc:1397] T 15a56323fe38453486c84ae761078bb2 P 0e7940e58fe64aa3a248e733e4e5ccea: Time spent bootstrapping tablet: real 0.017s user 0.009s sys 0.007s
I20260812 16:40:30.517482 30790 raft_consensus.cc:348] T 15a56323fe38453486c84ae761078bb2 P 0e7940e58fe64aa3a248e733e4e5ccea [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7940e58fe64aa3a248e733e4e5ccea" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 39003 } } peers { permanent_uuid: "d585036a34f74e2586fc4f8743561ea6" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44263 } } peers { permanent_uuid: "d400d8ef8d8c47128e143573073f6aa1" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 34789 } }
I20260812 16:40:30.517988 30790 raft_consensus.cc:729] T 15a56323fe38453486c84ae761078bb2 P 0e7940e58fe64aa3a248e733e4e5ccea [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e7940e58fe64aa3a248e733e4e5ccea, State: Initialized, Role: FOLLOWER
I20260812 16:40:30.518206 30790 consensus_queue.cc:260] T 15a56323fe38453486c84ae761078bb2 P 0e7940e58fe64aa3a248e733e4e5ccea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7940e58fe64aa3a248e733e4e5ccea" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 39003 } } peers { permanent_uuid: "d585036a34f74e2586fc4f8743561ea6" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44263 } } peers { permanent_uuid: "d400d8ef8d8c47128e143573073f6aa1" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 34789 } }
I20260812 16:40:30.518594 30790 ts_tablet_manager.cc:1428] T 15a56323fe38453486c84ae761078bb2 P 0e7940e58fe64aa3a248e733e4e5ccea: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
Aug 12 16:40:30 dist-test-slave-w4v5 krb5kdc[29431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552830, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:40:30.523677 30990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:30.508655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.193:34221)
0812 16:40:30.508839 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:30.508887 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 16:40:30.509652 (+ 765us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:40:30.511793 (+ 2141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:40:30.511798 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:40:30.512175 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:40:30.512433 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:40:30.512440 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:40:30.514064 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:40:30.514066 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:40:30.515404 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:40:30.515407 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:40:30.516223 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:40:30.516224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:40:30.516373 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:30.517165 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:40:30.517179 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:40:30.519201 (+ 2022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:40:30.521750 (+ 2549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:40:30.521757 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:40:30.521759 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:40:30.521999 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:40:30.522322 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:40:30.522323 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:40:30.522324 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:40:30.522441 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:40:30.523017 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:40:30.523023 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:40:30.523248 (+ 225us) client_negotiation.cc:769] Sending connection context
0812 16:40:30.523344 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 16:40:30.523487 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":86,"threads_started":1}
I20260812 16:40:30.523856 30992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:30.508756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:34221 (local address 127.0.253.254:39981)
0812 16:40:30.511229 (+ 2473us) server_negotiation.cc:206] Beginning negotiation
0812 16:40:30.511240 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:40:30.511294 (+ 54us) server_negotiation.cc:407] Connection header received
0812 16:40:30.511450 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:40:30.511457 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:40:30.511522 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:40:30.511738 (+ 216us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:40:30.512589 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:30.513932 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:30.515505 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:30.516103 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:30.516728 (+ 625us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:30.519315 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:40:30.519320 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:40:30.519320 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:40:30.519334 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:40:30.521644 (+ 2310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:40:30.522091 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:40:30.522095 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:40:30.522096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:40:30.522232 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:40:30.522521 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:40:30.522522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:40:30.522522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:40:30.522935 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:40:30.523038 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 16:40:30.523443 (+ 405us) server_negotiation.cc:299] Negotiation successful
0812 16:40:30.523611 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2413,"thread_start_us":89,"threads_started":1}
I20260812 16:40:30.524708 30988 heartbeater.cc:347] Connected to a master server at 127.0.253.254:39981
I20260812 16:40:30.524819 30988 heartbeater.cc:464] Registering TS with master...
I20260812 16:40:30.525007 30988 heartbeater.cc:511] Master 127.0.253.254:39981 requested a full tablet report, sending...
I20260812 16:40:30.525808 30640 ts_manager.cc:194] Registered new tserver with Master: 0e7940e58fe64aa3a248e733e4e5ccea (127.0.253.193:39003)
I20260812 16:40:30.527419 30640 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.193@KRBTEST.COM'} at 127.0.253.193:34221
I20260812 16:40:30.528321 30988 heartbeater.cc:503] Master 127.0.253.254:39981 was elected leader, sending a full tablet report...
Aug 12 16:40:30 dist-test-slave-w4v5 krb5kdc[29431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552830, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for kudu/127.0.253.193@KRBTEST.COM
I20260812 16:40:30.536165 31005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:30.526776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50568 (local address 127.0.253.193:39003)
0812 16:40:30.526995 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0812 16:40:30.527000 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:40:30.527011 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:40:30.527177 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:40:30.527179 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:40:30.527201 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:40:30.527281 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:40:30.527871 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:30.529178 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:30.530014 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:30.530570 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:30.530665 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:30.532912 (+ 2247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:40:30.532920 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:40:30.532921 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:40:30.532928 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:40:30.534795 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:40:30.535263 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:40:30.535266 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:40:30.535266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:40:30.535328 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:40:30.535529 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:40:30.535530 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:40:30.535530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:40:30.535688 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:40:30.535780 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 16:40:30.535975 (+ 195us) server_negotiation.cc:299] Negotiation successful
0812 16:40:30.536070 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":94,"threads_started":1}
I20260812 16:40:30.536166 31001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:30.526714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:39003 (local address 127.0.0.1:50568)
0812 16:40:30.526902 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:30.526939 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 16:40:30.527012 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:40:30.527282 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:40:30.527288 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:40:30.527607 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:40:30.527673 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:40:30.527675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:40:30.529287 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:40:30.529288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:40:30.529922 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:40:30.529924 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:40:30.530670 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:40:30.530671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:40:30.530746 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:30.531418 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:40:30.531433 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:40:30.532798 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:40:30.534890 (+ 2092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:40:30.534897 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:40:30.534899 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:40:30.535179 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:40:30.535396 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:40:30.535397 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:40:30.535397 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:40:30.535462 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:40:30.535771 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:40:30.535778 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:40:30.535835 (+ 57us) client_negotiation.cc:769] Sending connection context
0812 16:40:30.535921 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 16:40:30.536061 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":128,"threads_started":1}
I20260812 16:40:30.536551 30864 tablet_service.cc:1514] Processing DeleteTablet for tablet 15a56323fe38453486c84ae761078bb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:40:10 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:50568
I20260812 16:40:30.537088 31006 tablet_replica.cc:330] stopping tablet replica
I20260812 16:40:30.537206 31006 raft_consensus.cc:2229] T 15a56323fe38453486c84ae761078bb2 P 0e7940e58fe64aa3a248e733e4e5ccea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:40:30.537281 31006 raft_consensus.cc:2258] T 15a56323fe38453486c84ae761078bb2 P 0e7940e58fe64aa3a248e733e4e5ccea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:40:30.537642 31006 ts_tablet_manager.cc:1905] T 15a56323fe38453486c84ae761078bb2 P 0e7940e58fe64aa3a248e733e4e5ccea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:40:30.539664 31006 ts_tablet_manager.cc:1918] T 15a56323fe38453486c84ae761078bb2 P 0e7940e58fe64aa3a248e733e4e5ccea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:40:30.539732 31006 log.cc:1196] T 15a56323fe38453486c84ae761078bb2 P 0e7940e58fe64aa3a248e733e4e5ccea: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/wal/wals/15a56323fe38453486c84ae761078bb2
I20260812 16:40:30.539974 31006 ts_tablet_manager.cc:1939] T 15a56323fe38453486c84ae761078bb2 P 0e7940e58fe64aa3a248e733e4e5ccea: Deleting consensus metadata
I20260812 16:40:30.540494 30608 catalog_manager.cc:4841] TS 0e7940e58fe64aa3a248e733e4e5ccea (127.0.253.193:39003): tablet 15a56323fe38453486c84ae761078bb2 (table test-table [id=eb6d57d348754ea5be240110e4518749]) successfully deleted
W20260812 16:40:30.615095 30997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:40:30.615406 30997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:40:30.615450 30997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:40:30.616829 30997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:40:30.616919 30997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:40:30.616968 30997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.194
I20260812 16:40:30.618680 30997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:43519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.194:44263
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.253.194
--webserver_port=41709
--tserver_master_addrs=127.0.253.254:39981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.194
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:40:30.618999 30997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:40:30.619304 30997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:40:30.625013 31015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:40:30.625082 30997 server_base.cc:1034] running on GCE node
W20260812 16:40:30.625007 31013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:40:30.625008 31019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:40:30.625592 30997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:40:30.625825 30997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:40:30.626995 30997 hybrid_clock.cc:648] HybridClock initialized: now 1786552830626980 us; error 31 us; skew 500 ppm
Aug 12 16:40:30 dist-test-slave-w4v5 krb5kdc[29431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552830, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:40:30.629967 30997 init.cc:377] Logged in from keytab as kudu/127.0.253.194@KRBTEST.COM (short username kudu)
I20260812 16:40:30.630515 30997 webserver.cc:460] Webserver started at http://127.0.253.194:41709/ using document root <none> and password file <none>
I20260812 16:40:30.630789 30997 fs_manager.cc:359] Metadata directory not provided
I20260812 16:40:30.630890 30997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:40:30.632496 30997 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:40:30.633374 31037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:40:30.633549 30997 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:40:30.633665 30997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/wal
uuid: "d585036a34f74e2586fc4f8743561ea6"
format_stamp: "Formatted at 2026-08-12 16:40:09 on dist-test-slave-w4v5"
server_key: "632f75a515053f5d72a5e61f9a2da7bd"
server_key_iv: "3f3c2fe50fa19c104b8674e9b09c1ea8"
server_key_version: "encryptionkey@0"
I20260812 16:40:30.634080 30997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:40:30.652537 30997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:40:30.654778 30997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:40:30.654953 30997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:40:30.655261 30997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:40:30.655606 30997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:40:30.655668 30997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:40:30.655704 30997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:40:30.655731 30997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:40:30.661553 30997 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.194:44263
I20260812 16:40:30.661598 31259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.194:44263 every 8 connection(s)
I20260812 16:40:30.661950 30997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
I20260812 16:40:30.663689 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 30997
I20260812 16:40:30.664000 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.195:34789
--local_ip_for_outbound_sockets=127.0.253.195
--tserver_master_addrs=127.0.253.254:39981
--webserver_port=45275
--webserver_interface=127.0.253.195
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:43519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 16:40:30 dist-test-slave-w4v5 krb5kdc[29431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552830, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:40:30.672708 30992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:30.662688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:41501 (local address 127.0.253.254:39981)
0812 16:40:30.662775 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 16:40:30.662777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:40:30.662979 (+ 202us) server_negotiation.cc:407] Connection header received
0812 16:40:30.663529 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:40:30.663530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:40:30.663544 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:40:30.663635 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:40:30.664730 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:30.665486 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:30.666587 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:30.666825 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:30.666918 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:30.669488 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:40:30.669490 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:40:30.669490 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:40:30.669499 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:40:30.671244 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:40:30.671678 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:40:30.671679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:40:30.671680 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:40:30.671746 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:40:30.672000 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:40:30.672001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:40:30.672001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:40:30.672147 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:40:30.672221 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 16:40:30.672511 (+ 290us) server_negotiation.cc:299] Negotiation successful
0812 16:40:30.672610 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 16:40:30.672921 31263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:30.662682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.194:41501)
0812 16:40:30.662834 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:30.662876 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 16:40:30.663429 (+ 553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:40:30.663690 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:40:30.663694 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:40:30.664420 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:40:30.664623 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:40:30.664628 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:40:30.665585 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:40:30.665585 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:40:30.666490 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:40:30.666493 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:40:30.666932 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:40:30.666933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:40:30.667101 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:30.667613 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:40:30.667624 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:40:30.669380 (+ 1756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:40:30.671341 (+ 1961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:40:30.671350 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:40:30.671352 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:40:30.671596 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:40:30.671813 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:40:30.671814 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:40:30.671815 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:40:30.671931 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:40:30.672226 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:40:30.672233 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:40:30.672373 (+ 140us) client_negotiation.cc:769] Sending connection context
0812 16:40:30.672477 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 16:40:30.672660 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":80,"threads_started":1}
I20260812 16:40:30.673522 31260 heartbeater.cc:347] Connected to a master server at 127.0.253.254:39981
I20260812 16:40:30.673638 31260 heartbeater.cc:464] Registering TS with master...
I20260812 16:40:30.673867 31260 heartbeater.cc:511] Master 127.0.253.254:39981 requested a full tablet report, sending...
I20260812 16:40:30.674307 30640 ts_manager.cc:194] Registered new tserver with Master: d585036a34f74e2586fc4f8743561ea6 (127.0.253.194:44263)
I20260812 16:40:30.674690 30640 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.194@KRBTEST.COM'} at 127.0.253.194:41501
W20260812 16:40:30.752234 31266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:40:30.752482 31266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:40:30.752521 31266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:40:30.753677 31266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:40:30.753741 31266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:40:30.753783 31266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.195
I20260812 16:40:30.755306 31266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:43519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.195:34789
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.253.195
--webserver_port=45275
--tserver_master_addrs=127.0.253.254:39981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.195
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:40:30.755578 31266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:40:30.755823 31266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:40:30.763111 31274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:40:30.763264 31281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:40:30.763151 31277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:40:30.764400 31266 server_base.cc:1034] running on GCE node
I20260812 16:40:30.764628 31266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:40:30.764882 31266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:40:30.766052 31266 hybrid_clock.cc:648] HybridClock initialized: now 1786552830766039 us; error 30 us; skew 500 ppm
Aug 12 16:40:30 dist-test-slave-w4v5 krb5kdc[29431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552830, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:40:30.768687 31266 init.cc:377] Logged in from keytab as kudu/127.0.253.195@KRBTEST.COM (short username kudu)
I20260812 16:40:30.769098 31266 webserver.cc:460] Webserver started at http://127.0.253.195:45275/ using document root <none> and password file <none>
I20260812 16:40:30.769323 31266 fs_manager.cc:359] Metadata directory not provided
I20260812 16:40:30.769377 31266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:40:30.770808 31266 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:40:30.771509 31298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:40:30.771699 31266 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 16:40:30.771788 31266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/wal
uuid: "d400d8ef8d8c47128e143573073f6aa1"
format_stamp: "Formatted at 2026-08-12 16:40:09 on dist-test-slave-w4v5"
server_key: "90db868f2b1bd673f0d16f3b6c1c461c"
server_key_iv: "f2a504d32ec34742b4894c1bf1554f0c"
server_key_version: "encryptionkey@0"
I20260812 16:40:30.772135 31266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:40:30.789966 31266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:40:30.792858 31266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:40:30.792994 31266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:40:30.793224 31266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:40:30.793536 31266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:40:30.793570 31266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:40:30.793596 31266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:40:30.793608 31266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:40:30.799783 31266 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.195:34789
I20260812 16:40:30.799880 31528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.195:34789 every 8 connection(s)
I20260812 16:40:30.800232 31266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
I20260812 16:40:30.801282 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 31266
Aug 12 16:40:30 dist-test-slave-w4v5 krb5kdc[29431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552830, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:40:30.809764 31537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:30.801790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51154 (local address 127.0.253.254:39981)
0812 16:40:30.801925 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 16:40:30.801927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:40:30.801999 (+ 72us) server_negotiation.cc:407] Connection header received
0812 16:40:30.802062 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:40:30.802063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:40:30.802076 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:40:30.802188 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:40:30.802868 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:30.803755 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:30.804787 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:30.805110 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:30.805195 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:30.806705 (+ 1510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:40:30.806707 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:40:30.806707 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:40:30.806715 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:40:30.808456 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:40:30.808820 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:40:30.808821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:40:30.808822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:40:30.808887 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:40:30.809092 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:40:30.809094 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:40:30.809094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:40:30.809220 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:40:30.809425 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0812 16:40:30.809463 (+ 38us) server_negotiation.cc:299] Negotiation successful
0812 16:40:30.809692 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":79,"threads_started":1}
I20260812 16:40:30.812255 30992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:30.801143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:54987 (local address 127.0.253.254:39981)
0812 16:40:30.801236 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 16:40:30.801238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:40:30.801481 (+ 243us) server_negotiation.cc:407] Connection header received
0812 16:40:30.802219 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:40:30.802220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:40:30.802233 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:40:30.802312 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:40:30.803639 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:30.804344 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:30.805826 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:30.806147 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:30.806240 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:30.808922 (+ 2682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:40:30.808924 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:40:30.808925 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:40:30.808933 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:40:30.810699 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:40:30.811187 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:40:30.811188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:40:30.811189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:40:30.811254 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:40:30.811534 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:40:30.811535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:40:30.811535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:40:30.811662 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:40:30.811737 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 16:40:30.812054 (+ 317us) server_negotiation.cc:299] Negotiation successful
0812 16:40:30.812144 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 16:40:30.812700 31535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:30.801128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.195:54987)
0812 16:40:30.801340 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:30.801387 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 16:40:30.802108 (+ 721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:40:30.802382 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:40:30.802386 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:40:30.803229 (+ 843us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:40:30.803535 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:40:30.803542 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:40:30.804445 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:40:30.804445 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:40:30.805725 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:40:30.805728 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:40:30.806247 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:40:30.806248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:40:30.806358 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:30.806888 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:40:30.806898 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:40:30.808818 (+ 1920us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:40:30.810794 (+ 1976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:40:30.810802 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:40:30.810805 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:40:30.811059 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:40:30.811326 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:40:30.811327 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:40:30.811328 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:40:30.811456 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:40:30.811741 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:40:30.811749 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:40:30.811906 (+ 157us) client_negotiation.cc:769] Sending connection context
0812 16:40:30.812016 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 16:40:30.812225 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":125,"threads_started":1}
I20260812 16:40:30.813223 31532 heartbeater.cc:347] Connected to a master server at 127.0.253.254:39981
I20260812 16:40:30.813316 31532 heartbeater.cc:464] Registering TS with master...
I20260812 16:40:30.813529 31532 heartbeater.cc:511] Master 127.0.253.254:39981 requested a full tablet report, sending...
I20260812 16:40:30.813966 30640 ts_manager.cc:194] Registered new tserver with Master: d400d8ef8d8c47128e143573073f6aa1 (127.0.253.195:34789)
I20260812 16:40:30.814311 30640 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.195@KRBTEST.COM'} at 127.0.253.195:54987
I20260812 16:40:30.822005 1015 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:40:30.825189 30992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:30.822507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51164 (local address 127.0.253.254:39981)
0812 16:40:30.822575 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 16:40:30.822577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:40:30.822714 (+ 137us) server_negotiation.cc:407] Connection header received
0812 16:40:30.822758 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:40:30.822759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:40:30.822801 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:40:30.822873 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:40:30.823148 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:30.823657 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:30.824485 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:30.824722 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:30.824788 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:30.824937 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:40:30.824990 (+ 53us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0812 16:40:30.825132 (+ 142us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51164: token expired
Metrics: {"server-negotiator.queue_time_us":33}
W20260812 16:40:30.825245 30992 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51164: token expired
W20260812 16:40:30.825440 30506 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.253.254:39981)
I20260812 16:40:30.833196 30992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:30.825776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51166 (local address 127.0.253.254:39981)
0812 16:40:30.825815 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 16:40:30.825817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:40:30.825824 (+ 7us) server_negotiation.cc:407] Connection header received
0812 16:40:30.825849 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:40:30.825850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:40:30.825858 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:40:30.825911 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:40:30.826478 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:30.826997 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:30.827955 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:30.828140 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:30.828209 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:30.829648 (+ 1439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:40:30.829650 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:40:30.829650 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:40:30.829659 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:40:30.831390 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:40:30.831937 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:40:30.831939 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:40:30.831939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:40:30.832003 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:40:30.832461 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:40:30.832463 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:40:30.832463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:40:30.832595 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:40:30.832726 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 16:40:30.832990 (+ 264us) server_negotiation.cc:299] Negotiation successful
0812 16:40:30.833116 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 16:40:30.834839 30640 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51166:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:40:30.836922 30640 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:40:30.839826 30864 tablet_service.cc:1467] Processing CreateTablet for tablet b84942e43af64d09a6fbffb23c9ba263 (DEFAULT_TABLE table=test-table [id=0d041aaced1d43199bdc36486b9572cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:40:30.840109 30864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b84942e43af64d09a6fbffb23c9ba263. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:40:30.841660 30790 tablet_bootstrap.cc:492] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea: Bootstrap starting.
I20260812 16:40:30.842089 30790 tablet_bootstrap.cc:654] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea: Neither blocks nor log segments found. Creating new log.
I20260812 16:40:30.843600 30790 tablet_bootstrap.cc:492] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea: No bootstrap required, opened a new log
I20260812 16:40:30.843674 30790 ts_tablet_manager.cc:1397] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:40:30.843852 30790 raft_consensus.cc:348] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d585036a34f74e2586fc4f8743561ea6" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44263 } } peers { permanent_uuid: "d400d8ef8d8c47128e143573073f6aa1" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 34789 } } peers { permanent_uuid: "0e7940e58fe64aa3a248e733e4e5ccea" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 39003 } }
I20260812 16:40:30.843957 30790 raft_consensus.cc:374] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:40:30.844035 30790 raft_consensus.cc:729] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e7940e58fe64aa3a248e733e4e5ccea, State: Initialized, Role: FOLLOWER
I20260812 16:40:30.844089 30790 consensus_queue.cc:260] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d585036a34f74e2586fc4f8743561ea6" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44263 } } peers { permanent_uuid: "d400d8ef8d8c47128e143573073f6aa1" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 34789 } } peers { permanent_uuid: "0e7940e58fe64aa3a248e733e4e5ccea" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 39003 } }
I20260812 16:40:30.844205 30790 ts_tablet_manager.cc:1428] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 16:40:30.844302 31001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:30.839489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:44263 (local address 127.0.0.1:56734)
0812 16:40:30.839534 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:30.839546 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:40:30.839590 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:40:30.840442 (+ 852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:40:30.840443 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:40:30.840445 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:40:30.840509 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:40:30.840511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:40:30.841517 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:40:30.841518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:40:30.842977 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:40:30.842980 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:40:30.843998 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:40:30.843999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:40:30.844084 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:30.844113 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 16:40:30.844150 (+ 37us) client_negotiation.cc:240] Negotiation successful
0812 16:40:30.844220 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260812 16:40:30.844898 31552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:30.839852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56734 (local address 127.0.253.194:44263)
0812 16:40:30.840247 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0812 16:40:30.840253 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:40:30.840265 (+ 12us) server_negotiation.cc:407] Connection header received
0812 16:40:30.840312 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:40:30.840314 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:40:30.840330 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:40:30.840435 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:40:30.840582 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:30.841407 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:30.843054 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:30.843885 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:30.844613 (+ 728us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:30.844699 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 16:40:30.844735 (+ 36us) server_negotiation.cc:299] Negotiation successful
0812 16:40:30.844786 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":153,"threads_started":1}
I20260812 16:40:30.845837 31136 tablet_service.cc:1467] Processing CreateTablet for tablet b84942e43af64d09a6fbffb23c9ba263 (DEFAULT_TABLE table=test-table [id=0d041aaced1d43199bdc36486b9572cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:40:30.846170 31550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:30.839590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.195:34789 (local address 127.0.0.1:35380)
0812 16:40:30.840127 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:30.840173 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 16:40:30.840248 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:40:30.842260 (+ 2012us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:40:30.842261 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:40:30.842263 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:40:30.842306 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:40:30.842308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:40:30.843775 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:40:30.843776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:40:30.844977 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:40:30.844982 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:40:30.845869 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:40:30.845870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:40:30.845942 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:30.845947 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:40:30.846025 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 16:40:30.846077 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":138,"threads_started":1}
I20260812 16:40:30.846187 31554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:30.841767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35380 (local address 127.0.253.195:34789)
0812 16:40:30.842009 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0812 16:40:30.842014 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:40:30.842028 (+ 14us) server_negotiation.cc:407] Connection header received
0812 16:40:30.842103 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:40:30.842105 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:40:30.842125 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:40:30.842240 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:40:30.842395 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:30.843665 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:30.845079 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:30.845779 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:30.845841 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:30.845880 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0812 16:40:30.846033 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 16:40:30.846096 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":108,"threads_started":1}
I20260812 16:40:30.846298 31136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b84942e43af64d09a6fbffb23c9ba263. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:40:30.846982 31383 tablet_service.cc:1467] Processing CreateTablet for tablet b84942e43af64d09a6fbffb23c9ba263 (DEFAULT_TABLE table=test-table [id=0d041aaced1d43199bdc36486b9572cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:40:30.847527 31383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b84942e43af64d09a6fbffb23c9ba263. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:40:30.850718 31557 tablet_bootstrap.cc:492] T b84942e43af64d09a6fbffb23c9ba263 P d585036a34f74e2586fc4f8743561ea6: Bootstrap starting.
I20260812 16:40:30.851369 31558 tablet_bootstrap.cc:492] T b84942e43af64d09a6fbffb23c9ba263 P d400d8ef8d8c47128e143573073f6aa1: Bootstrap starting.
I20260812 16:40:30.851539 31557 tablet_bootstrap.cc:654] T b84942e43af64d09a6fbffb23c9ba263 P d585036a34f74e2586fc4f8743561ea6: Neither blocks nor log segments found. Creating new log.
I20260812 16:40:30.852006 31558 tablet_bootstrap.cc:654] T b84942e43af64d09a6fbffb23c9ba263 P d400d8ef8d8c47128e143573073f6aa1: Neither blocks nor log segments found. Creating new log.
I20260812 16:40:30.852118 31557 log.cc:824] T b84942e43af64d09a6fbffb23c9ba263 P d585036a34f74e2586fc4f8743561ea6: Log is configured to *not* fsync() on all Append() calls
I20260812 16:40:30.852548 31558 log.cc:824] T b84942e43af64d09a6fbffb23c9ba263 P d400d8ef8d8c47128e143573073f6aa1: Log is configured to *not* fsync() on all Append() calls
I20260812 16:40:30.852869 31557 tablet_bootstrap.cc:492] T b84942e43af64d09a6fbffb23c9ba263 P d585036a34f74e2586fc4f8743561ea6: No bootstrap required, opened a new log
I20260812 16:40:30.852950 31557 ts_tablet_manager.cc:1397] T b84942e43af64d09a6fbffb23c9ba263 P d585036a34f74e2586fc4f8743561ea6: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 16:40:30.853253 31558 tablet_bootstrap.cc:492] T b84942e43af64d09a6fbffb23c9ba263 P d400d8ef8d8c47128e143573073f6aa1: No bootstrap required, opened a new log
I20260812 16:40:30.853312 31558 ts_tablet_manager.cc:1397] T b84942e43af64d09a6fbffb23c9ba263 P d400d8ef8d8c47128e143573073f6aa1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:40:30.854676 31557 raft_consensus.cc:348] T b84942e43af64d09a6fbffb23c9ba263 P d585036a34f74e2586fc4f8743561ea6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d585036a34f74e2586fc4f8743561ea6" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44263 } } peers { permanent_uuid: "d400d8ef8d8c47128e143573073f6aa1" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 34789 } } peers { permanent_uuid: "0e7940e58fe64aa3a248e733e4e5ccea" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 39003 } }
I20260812 16:40:30.854849 31557 raft_consensus.cc:374] T b84942e43af64d09a6fbffb23c9ba263 P d585036a34f74e2586fc4f8743561ea6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:40:30.854773 31558 raft_consensus.cc:348] T b84942e43af64d09a6fbffb23c9ba263 P d400d8ef8d8c47128e143573073f6aa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d585036a34f74e2586fc4f8743561ea6" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44263 } } peers { permanent_uuid: "d400d8ef8d8c47128e143573073f6aa1" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 34789 } } peers { permanent_uuid: "0e7940e58fe64aa3a248e733e4e5ccea" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 39003 } }
I20260812 16:40:30.854898 31557 raft_consensus.cc:729] T b84942e43af64d09a6fbffb23c9ba263 P d585036a34f74e2586fc4f8743561ea6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d585036a34f74e2586fc4f8743561ea6, State: Initialized, Role: FOLLOWER
I20260812 16:40:30.854939 31558 raft_consensus.cc:374] T b84942e43af64d09a6fbffb23c9ba263 P d400d8ef8d8c47128e143573073f6aa1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:40:30.854987 31558 raft_consensus.cc:729] T b84942e43af64d09a6fbffb23c9ba263 P d400d8ef8d8c47128e143573073f6aa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d400d8ef8d8c47128e143573073f6aa1, State: Initialized, Role: FOLLOWER
I20260812 16:40:30.855062 31557 consensus_queue.cc:260] T b84942e43af64d09a6fbffb23c9ba263 P d585036a34f74e2586fc4f8743561ea6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d585036a34f74e2586fc4f8743561ea6" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44263 } } peers { permanent_uuid: "d400d8ef8d8c47128e143573073f6aa1" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 34789 } } peers { permanent_uuid: "0e7940e58fe64aa3a248e733e4e5ccea" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 39003 } }
I20260812 16:40:30.855161 31558 consensus_queue.cc:260] T b84942e43af64d09a6fbffb23c9ba263 P d400d8ef8d8c47128e143573073f6aa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d585036a34f74e2586fc4f8743561ea6" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44263 } } peers { permanent_uuid: "d400d8ef8d8c47128e143573073f6aa1" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 34789 } } peers { permanent_uuid: "0e7940e58fe64aa3a248e733e4e5ccea" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 39003 } }
I20260812 16:40:30.855381 31557 ts_tablet_manager.cc:1428] T b84942e43af64d09a6fbffb23c9ba263 P d585036a34f74e2586fc4f8743561ea6: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 16:40:30.855485 31532 heartbeater.cc:503] Master 127.0.253.254:39981 was elected leader, sending a full tablet report...
I20260812 16:40:30.855520 31260 heartbeater.cc:503] Master 127.0.253.254:39981 was elected leader, sending a full tablet report...
I20260812 16:40:30.855749 31558 ts_tablet_manager.cc:1428] T b84942e43af64d09a6fbffb23c9ba263 P d400d8ef8d8c47128e143573073f6aa1: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 16:40:30.864400 31562 raft_consensus.cc:482] T b84942e43af64d09a6fbffb23c9ba263 P d585036a34f74e2586fc4f8743561ea6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:40:30.864509 31562 raft_consensus.cc:504] T b84942e43af64d09a6fbffb23c9ba263 P d585036a34f74e2586fc4f8743561ea6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d585036a34f74e2586fc4f8743561ea6" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44263 } } peers { permanent_uuid: "d400d8ef8d8c47128e143573073f6aa1" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 34789 } } peers { permanent_uuid: "0e7940e58fe64aa3a248e733e4e5ccea" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 39003 } }
I20260812 16:40:30.864971 31562 leader_election.cc:288] T b84942e43af64d09a6fbffb23c9ba263 P d585036a34f74e2586fc4f8743561ea6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d400d8ef8d8c47128e143573073f6aa1 (127.0.253.195:34789), 0e7940e58fe64aa3a248e733e4e5ccea (127.0.253.193:39003)
I20260812 16:40:30.869092 31005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:30.865251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:54737 (local address 127.0.253.193:39003)
0812 16:40:30.865325 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 16:40:30.865327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:40:30.865336 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:40:30.865366 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:40:30.865367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:40:30.865397 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:40:30.865466 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:40:30.865658 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:30.866464 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:30.867987 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:30.868616 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:30.868704 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:30.868794 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 16:40:30.868964 (+ 170us) server_negotiation.cc:299] Negotiation successful
0812 16:40:30.869018 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 16:40:30.869092 31554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:30.865327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:41769 (local address 127.0.253.195:34789)
0812 16:40:30.865376 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 16:40:30.865378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:40:30.865436 (+ 58us) server_negotiation.cc:407] Connection header received
0812 16:40:30.865530 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:40:30.865531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:40:30.865538 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:40:30.865580 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:40:30.865735 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:30.866467 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:30.867987 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:30.868616 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:30.868704 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:30.868714 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 16:40:30.868968 (+ 254us) server_negotiation.cc:299] Negotiation successful
0812 16:40:30.869016 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 16:40:30.869134 31566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:30.865180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.195:34789 (local address 127.0.253.194:41769)
0812 16:40:30.865370 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:30.865380 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:40:30.865467 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:40:30.865596 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:40:30.865597 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:40:30.865599 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:40:30.865665 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:40:30.865667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:40:30.866560 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:40:30.866561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:40:30.867868 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:40:30.867872 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:40:30.868730 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:40:30.868731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:40:30.868833 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:30.868838 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:40:30.868918 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 16:40:30.868986 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":83,"thread_start_us":117,"threads_started":1}
I20260812 16:40:30.869143 31263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:30.865160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:39003 (local address 127.0.253.194:54737)
0812 16:40:30.865226 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:30.865238 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:40:30.865282 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:40:30.865486 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:40:30.865486 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:40:30.865488 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:40:30.865581 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:40:30.865585 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:40:30.866560 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:40:30.866561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:40:30.867868 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:40:30.867872 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:40:30.868729 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:40:30.868731 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:40:30.868836 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:30.868838 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 16:40:30.868918 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 16:40:30.868986 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":57}
I20260812 16:40:30.869504 31437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b84942e43af64d09a6fbffb23c9ba263" candidate_uuid: "d585036a34f74e2586fc4f8743561ea6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d400d8ef8d8c47128e143573073f6aa1" is_pre_election: true
I20260812 16:40:30.869705 31437 raft_consensus.cc:2454] T b84942e43af64d09a6fbffb23c9ba263 P d400d8ef8d8c47128e143573073f6aa1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d585036a34f74e2586fc4f8743561ea6 in term 0.
I20260812 16:40:30.869684 30901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b84942e43af64d09a6fbffb23c9ba263" candidate_uuid: "d585036a34f74e2586fc4f8743561ea6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e7940e58fe64aa3a248e733e4e5ccea" is_pre_election: true
I20260812 16:40:30.869823 30901 raft_consensus.cc:2454] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d585036a34f74e2586fc4f8743561ea6 in term 0.
I20260812 16:40:30.869999 31043 leader_election.cc:302] T b84942e43af64d09a6fbffb23c9ba263 P d585036a34f74e2586fc4f8743561ea6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d400d8ef8d8c47128e143573073f6aa1, d585036a34f74e2586fc4f8743561ea6; no voters:
I20260812 16:40:30.870141 31562 raft_consensus.cc:2781] T b84942e43af64d09a6fbffb23c9ba263 P d585036a34f74e2586fc4f8743561ea6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:40:30.870213 31562 raft_consensus.cc:482] T b84942e43af64d09a6fbffb23c9ba263 P d585036a34f74e2586fc4f8743561ea6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:40:30.870232 31562 raft_consensus.cc:3037] T b84942e43af64d09a6fbffb23c9ba263 P d585036a34f74e2586fc4f8743561ea6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:40:30.871001 31562 raft_consensus.cc:504] T b84942e43af64d09a6fbffb23c9ba263 P d585036a34f74e2586fc4f8743561ea6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d585036a34f74e2586fc4f8743561ea6" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44263 } } peers { permanent_uuid: "d400d8ef8d8c47128e143573073f6aa1" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 34789 } } peers { permanent_uuid: "0e7940e58fe64aa3a248e733e4e5ccea" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 39003 } }
I20260812 16:40:30.871186 31562 leader_election.cc:288] T b84942e43af64d09a6fbffb23c9ba263 P d585036a34f74e2586fc4f8743561ea6 [CANDIDATE]: Term 1 election: Requested vote from peers d400d8ef8d8c47128e143573073f6aa1 (127.0.253.195:34789), 0e7940e58fe64aa3a248e733e4e5ccea (127.0.253.193:39003)
I20260812 16:40:30.871352 30901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b84942e43af64d09a6fbffb23c9ba263" candidate_uuid: "d585036a34f74e2586fc4f8743561ea6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e7940e58fe64aa3a248e733e4e5ccea"
I20260812 16:40:30.871402 31437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b84942e43af64d09a6fbffb23c9ba263" candidate_uuid: "d585036a34f74e2586fc4f8743561ea6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d400d8ef8d8c47128e143573073f6aa1"
I20260812 16:40:30.871455 30901 raft_consensus.cc:3037] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:40:30.871491 31437 raft_consensus.cc:3037] T b84942e43af64d09a6fbffb23c9ba263 P d400d8ef8d8c47128e143573073f6aa1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:40:30.872267 31437 raft_consensus.cc:2454] T b84942e43af64d09a6fbffb23c9ba263 P d400d8ef8d8c47128e143573073f6aa1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d585036a34f74e2586fc4f8743561ea6 in term 1.
I20260812 16:40:30.872324 30901 raft_consensus.cc:2454] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d585036a34f74e2586fc4f8743561ea6 in term 1.
I20260812 16:40:30.872483 31043 leader_election.cc:302] T b84942e43af64d09a6fbffb23c9ba263 P d585036a34f74e2586fc4f8743561ea6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d400d8ef8d8c47128e143573073f6aa1, d585036a34f74e2586fc4f8743561ea6; no voters:
I20260812 16:40:30.872597 31562 raft_consensus.cc:2781] T b84942e43af64d09a6fbffb23c9ba263 P d585036a34f74e2586fc4f8743561ea6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:40:30.872762 31562 raft_consensus.cc:686] T b84942e43af64d09a6fbffb23c9ba263 P d585036a34f74e2586fc4f8743561ea6 [term 1 LEADER]: Becoming Leader. State: Replica: d585036a34f74e2586fc4f8743561ea6, State: Running, Role: LEADER
I20260812 16:40:30.872867 31562 consensus_queue.cc:237] T b84942e43af64d09a6fbffb23c9ba263 P d585036a34f74e2586fc4f8743561ea6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d585036a34f74e2586fc4f8743561ea6" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44263 } } peers { permanent_uuid: "d400d8ef8d8c47128e143573073f6aa1" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 34789 } } peers { permanent_uuid: "0e7940e58fe64aa3a248e733e4e5ccea" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 39003 } }
I20260812 16:40:30.873878 30638 catalog_manager.cc:5495] T b84942e43af64d09a6fbffb23c9ba263 P d585036a34f74e2586fc4f8743561ea6 reported cstate change: term changed from 0 to 1, leader changed from <none> to d585036a34f74e2586fc4f8743561ea6 (127.0.253.194). New cstate: current_term: 1 leader_uuid: "d585036a34f74e2586fc4f8743561ea6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d585036a34f74e2586fc4f8743561ea6" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d400d8ef8d8c47128e143573073f6aa1" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 34789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e7940e58fe64aa3a248e733e4e5ccea" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 39003 } health_report { overall_health: UNKNOWN } } }
I20260812 16:40:30.882601 31552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:30.880057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56742 (local address 127.0.253.194:44263)
0812 16:40:30.880157 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 16:40:30.880159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:40:30.880168 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:40:30.880201 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:40:30.880201 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:40:30.880236 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:40:30.880341 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:40:30.880488 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:30.880980 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:30.881808 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:30.882002 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:30.882074 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:30.882244 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:40:30.882350 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:40:30.882438 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 16:40:30.882461 (+ 23us) server_negotiation.cc:299] Negotiation successful
0812 16:40:30.882483 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 16:40:30.884596 31437 raft_consensus.cc:1261] T b84942e43af64d09a6fbffb23c9ba263 P d400d8ef8d8c47128e143573073f6aa1 [term 1 FOLLOWER]: Refusing update from remote peer d585036a34f74e2586fc4f8743561ea6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:40:30.884613 30901 raft_consensus.cc:1261] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [term 1 FOLLOWER]: Refusing update from remote peer d585036a34f74e2586fc4f8743561ea6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:40:30.885094 31562 consensus_queue.cc:1035] T b84942e43af64d09a6fbffb23c9ba263 P d585036a34f74e2586fc4f8743561ea6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d400d8ef8d8c47128e143573073f6aa1" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 34789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:40:30.885315 31574 consensus_queue.cc:1035] T b84942e43af64d09a6fbffb23c9ba263 P d585036a34f74e2586fc4f8743561ea6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e7940e58fe64aa3a248e733e4e5ccea" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:40:30.887698 31577 mvcc.cc:205] Tried to move back new op lower bound from 7317720395300397056 to 7317720395256131584. Current Snapshot: MvccSnapshot[applied={T|T < 7317720395300397056}]
I20260812 16:40:30.888188 31583 mvcc.cc:205] Tried to move back new op lower bound from 7317720395300397056 to 7317720395256131584. Current Snapshot: MvccSnapshot[applied={T|T < 7317720395300397056}]
I20260812 16:40:30.888746 31587 mvcc.cc:205] Tried to move back new op lower bound from 7317720395300397056 to 7317720395256131584. Current Snapshot: MvccSnapshot[applied={T|T < 7317720395300397056}]
I20260812 16:40:30.890498 30638 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:40:30.890586 30638 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:40:30.891482 30638 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea: Sending DeleteTablet for 3 replicas of tablet b84942e43af64d09a6fbffb23c9ba263
I20260812 16:40:30.891878 31383 tablet_service.cc:1514] Processing DeleteTablet for tablet b84942e43af64d09a6fbffb23c9ba263 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:40:30 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:35380
I20260812 16:40:30.891860 31136 tablet_service.cc:1514] Processing DeleteTablet for tablet b84942e43af64d09a6fbffb23c9ba263 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:40:30 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:56734
I20260812 16:40:30.892081 30864 tablet_service.cc:1514] Processing DeleteTablet for tablet b84942e43af64d09a6fbffb23c9ba263 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:40:30 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:50568
I20260812 16:40:30.892135 31602 tablet_replica.cc:330] T b84942e43af64d09a6fbffb23c9ba263 P d585036a34f74e2586fc4f8743561ea6: stopping tablet replica
I20260812 16:40:30.892179 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 30707
I20260812 16:40:30.892266 31602 raft_consensus.cc:2229] T b84942e43af64d09a6fbffb23c9ba263 P d585036a34f74e2586fc4f8743561ea6 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:40:30.892287 31006 tablet_replica.cc:330] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea: stopping tablet replica
I20260812 16:40:30.892344 31006 raft_consensus.cc:2229] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:40:30.892365 31602 raft_consensus.cc:2258] T b84942e43af64d09a6fbffb23c9ba263 P d585036a34f74e2586fc4f8743561ea6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:40:30.892401 31006 raft_consensus.cc:2258] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:40:30.900646 31602 ts_tablet_manager.cc:1905] T b84942e43af64d09a6fbffb23c9ba263 P d585036a34f74e2586fc4f8743561ea6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:40:30.902699 31602 ts_tablet_manager.cc:1918] T b84942e43af64d09a6fbffb23c9ba263 P d585036a34f74e2586fc4f8743561ea6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:40:30.902793 31602 log.cc:1196] T b84942e43af64d09a6fbffb23c9ba263 P d585036a34f74e2586fc4f8743561ea6: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/wal/wals/b84942e43af64d09a6fbffb23c9ba263
I20260812 16:40:30.903146 31602 ts_tablet_manager.cc:1939] T b84942e43af64d09a6fbffb23c9ba263 P d585036a34f74e2586fc4f8743561ea6: Deleting consensus metadata
I20260812 16:40:30.903708 30618 catalog_manager.cc:4841] TS d585036a34f74e2586fc4f8743561ea6 (127.0.253.194:44263): tablet b84942e43af64d09a6fbffb23c9ba263 (table test-table [id=0d041aaced1d43199bdc36486b9572cc]) successfully deleted
I20260812 16:40:30.915889 31603 tablet_replica.cc:330] T b84942e43af64d09a6fbffb23c9ba263 P d400d8ef8d8c47128e143573073f6aa1: stopping tablet replica
I20260812 16:40:30.916100 31603 raft_consensus.cc:2229] T b84942e43af64d09a6fbffb23c9ba263 P d400d8ef8d8c47128e143573073f6aa1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:40:30.916299 31603 raft_consensus.cc:2258] T b84942e43af64d09a6fbffb23c9ba263 P d400d8ef8d8c47128e143573073f6aa1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:40:30.919370 31603 ts_tablet_manager.cc:1905] T b84942e43af64d09a6fbffb23c9ba263 P d400d8ef8d8c47128e143573073f6aa1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:40:30.921639 31603 ts_tablet_manager.cc:1918] T b84942e43af64d09a6fbffb23c9ba263 P d400d8ef8d8c47128e143573073f6aa1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:40:30.921785 31603 log.cc:1196] T b84942e43af64d09a6fbffb23c9ba263 P d400d8ef8d8c47128e143573073f6aa1: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/wal/wals/b84942e43af64d09a6fbffb23c9ba263
I20260812 16:40:30.922125 31603 ts_tablet_manager.cc:1939] T b84942e43af64d09a6fbffb23c9ba263 P d400d8ef8d8c47128e143573073f6aa1: Deleting consensus metadata
I20260812 16:40:30.922870 30618 catalog_manager.cc:4841] TS d400d8ef8d8c47128e143573073f6aa1 (127.0.253.195:34789): tablet b84942e43af64d09a6fbffb23c9ba263 (table test-table [id=0d041aaced1d43199bdc36486b9572cc]) successfully deleted
I20260812 16:40:30.929939 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 30997
W20260812 16:40:30.930027 30608 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.253.193:39003 (error 108)
I20260812 16:40:30.939493 31550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:30.939300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:39003 (local address 127.0.0.1:50582)
0812 16:40:30.939372 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:30.939437 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.193:39003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 16:40:30.939714 30608 catalog_manager.cc:4584] TS 0e7940e58fe64aa3a248e733e4e5ccea (127.0.253.193:39003): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b84942e43af64d09a6fbffb23c9ba263: Network error: Client connection negotiation failed: client connection to 127.0.253.193:39003: connect: Connection refused (error 111)
I20260812 16:40:30.944034 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 31266
I20260812 16:40:30.960755 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 30567
I20260812 16:40:50.973171 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:39981
--webserver_interface=127.0.253.254
--webserver_port=33263
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:43519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:39981
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:40:51.058494 31605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:40:51.058763 31605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:40:51.058807 31605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:40:51.060101 31605 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 16:40:51.060179 31605 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 16:40:51.060204 31605 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 16:40:51.060235 31605 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:40:51.060263 31605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:40:51.060290 31605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:40:51.060312 31605 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:40:51.060343 31605 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:40:51.061909 31605 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:43519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:39981
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:39981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=33263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:40:51.062206 31605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:40:51.062451 31605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:40:51.067996 31614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:40:51.067997 31618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:40:51.068038 31605 server_base.cc:1034] running on GCE node
W20260812 16:40:51.068037 31624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:40:51.068462 31605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:40:51.068742 31605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:40:51.069938 31605 hybrid_clock.cc:648] HybridClock initialized: now 1786552851069932 us; error 44 us; skew 500 ppm
Aug 12 16:40:51 dist-test-slave-w4v5 krb5kdc[29431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552851, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:40:51.072714 31605 init.cc:377] Logged in from keytab as kudu/127.0.253.254@KRBTEST.COM (short username kudu)
I20260812 16:40:51.073223 31605 webserver.cc:460] Webserver started at http://127.0.253.254:33263/ using document root <none> and password file <none>
I20260812 16:40:51.073480 31605 fs_manager.cc:359] Metadata directory not provided
I20260812 16:40:51.073536 31605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:40:51.075003 31605 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:40:51.075855 31640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:40:51.076073 31605 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:40:51.076171 31605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "3bddd07e41ca4d92aaf62a0b496011ea"
format_stamp: "Formatted at 2026-08-12 16:40:09 on dist-test-slave-w4v5"
server_key: "ba0b99e8997ca623764cf5d495c46212"
server_key_iv: "ae6fc5547a948212895046dc3e8bed98"
server_key_version: "encryptionkey@0"
I20260812 16:40:51.076535 31605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:40:51.121088 31605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:40:51.123448 31605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:40:51.123606 31605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:40:51.128193 31605 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:39981
I20260812 16:40:51.128320 31750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:39981 every 8 connection(s)
I20260812 16:40:51.128705 31605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:40:51.130160 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 31605
I20260812 16:40:51.130520 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.193:39003
--local_ip_for_outbound_sockets=127.0.253.193
--tserver_master_addrs=127.0.253.254:39981
--webserver_port=46433
--webserver_interface=127.0.253.193
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:43519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:40:51.130609 31752 sys_catalog.cc:263] Verifying existing consensus state
I20260812 16:40:51.131789 31752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea: Bootstrap starting.
I20260812 16:40:51.141319 31752 log.cc:824] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea: Log is configured to *not* fsync() on all Append() calls
I20260812 16:40:51.143822 31752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 16:40:51.144168 31752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea: Bootstrap complete.
I20260812 16:40:51.146552 31752 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bddd07e41ca4d92aaf62a0b496011ea" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39981 } }
I20260812 16:40:51.146811 31752 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bddd07e41ca4d92aaf62a0b496011ea, State: Initialized, Role: FOLLOWER
I20260812 16:40:51.147027 31752 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bddd07e41ca4d92aaf62a0b496011ea" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39981 } }
I20260812 16:40:51.147137 31752 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:40:51.147195 31752 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:40:51.147244 31752 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [term 2 FOLLOWER]: Advancing to term 3
I20260812 16:40:51.148260 31752 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bddd07e41ca4d92aaf62a0b496011ea" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39981 } }
I20260812 16:40:51.148386 31752 leader_election.cc:302] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3bddd07e41ca4d92aaf62a0b496011ea; no voters:
I20260812 16:40:51.148612 31752 leader_election.cc:288] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 16:40:51.148648 31759 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [term 3 FOLLOWER]: Leader election won for term 3
I20260812 16:40:51.148772 31759 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [term 3 LEADER]: Becoming Leader. State: Replica: 3bddd07e41ca4d92aaf62a0b496011ea, State: Running, Role: LEADER
I20260812 16:40:51.148864 31759 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bddd07e41ca4d92aaf62a0b496011ea" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39981 } }
I20260812 16:40:51.148938 31752 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:40:51.149138 31764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3bddd07e41ca4d92aaf62a0b496011ea. Latest consensus state: current_term: 3 leader_uuid: "3bddd07e41ca4d92aaf62a0b496011ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bddd07e41ca4d92aaf62a0b496011ea" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39981 } } }
I20260812 16:40:51.149354 31764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [sys.catalog]: This master's current role is: LEADER
I20260812 16:40:51.149295 31761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "3bddd07e41ca4d92aaf62a0b496011ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bddd07e41ca4d92aaf62a0b496011ea" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 39981 } } }
I20260812 16:40:51.149621 31761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea [sys.catalog]: This master's current role is: LEADER
I20260812 16:40:51.149751 31781 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:40:51.150713 31781 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:40:51.150836 31781 catalog_manager.cc:1259] Loaded cluster ID: 7580f6e5149941e8865a4353b78d6187
I20260812 16:40:51.150876 31781 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:40:51.151932 31781 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:40:51.152154 31781 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea: Loaded TSK: 0
I20260812 16:40:51.152215 31781 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3bddd07e41ca4d92aaf62a0b496011ea: Loaded TSK: 1
F20260812 16:40:51.152442 31781 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f7e6ddb49ad google::LogMessage::Fail() at ??:0
@ 0x7f7e6ddb788a google::LogMessage::SendToLog() at ??:0
@ 0x7f7e6ddb44de google::LogMessage::Flush() at ??:0
@ 0x7f7e6ddb5a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f7e70521104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f7e7051ed41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f7e75d0e1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f7e75d0e48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f7e75d0e594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f7e75cd4c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f7e75cdead0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f7e6fc820d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f7e6fc77954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f7e70adb6db start_thread at ??:0
@ 0x7f7e6c8b261f clone at ??:0
W20260812 16:40:51.237931 31754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:40:51.238143 31754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:40:51.238168 31754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:40:51.239423 31754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:40:51.239470 31754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:40:51.239497 31754 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.193
I20260812 16:40:51.241022 31754 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:43519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.193:39003
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.253.193
--webserver_port=46433
--tserver_master_addrs=127.0.253.254:39981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.193
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:40:51.241302 31754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:40:51.241580 31754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:40:51.247921 31859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:40:51.247921 31852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:40:51.248226 31853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:40:51.248248 31754 server_base.cc:1034] running on GCE node
I20260812 16:40:51.248513 31754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:40:51.248798 31754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:40:51.249963 31754 hybrid_clock.cc:648] HybridClock initialized: now 1786552851249951 us; error 30 us; skew 500 ppm
Aug 12 16:40:51 dist-test-slave-w4v5 krb5kdc[29431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552851, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:40:51.252988 31754 init.cc:377] Logged in from keytab as kudu/127.0.253.193@KRBTEST.COM (short username kudu)
I20260812 16:40:51.253644 31754 webserver.cc:460] Webserver started at http://127.0.253.193:46433/ using document root <none> and password file <none>
I20260812 16:40:51.253957 31754 fs_manager.cc:359] Metadata directory not provided
I20260812 16:40:51.254025 31754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:40:51.256024 31754 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:40:51.257005 31878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:40:51.257215 31754 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:40:51.257311 31754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/wal
uuid: "0e7940e58fe64aa3a248e733e4e5ccea"
format_stamp: "Formatted at 2026-08-12 16:40:09 on dist-test-slave-w4v5"
server_key: "277a689f10147fc0e9b471a980e66894"
server_key_iv: "d58bb779abd4e0042bb7dca4fff0fa46"
server_key_version: "encryptionkey@0"
I20260812 16:40:51.257671 31754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:40:51.273371 31754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:40:51.275969 31754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:40:51.276124 31754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:40:51.276400 31754 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:40:51.276907 31889 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 16:40:51.279047 31754 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 16:40:51.279167 31754 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 16:40:51.279218 31754 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 16:40:51.279958 31754 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 16:40:51.280018 31754 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 16:40:51.280102 31889 tablet_bootstrap.cc:492] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea: Bootstrap starting.
I20260812 16:40:51.286564 31754 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.193:39003
I20260812 16:40:51.286746 32129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.193:39003 every 8 connection(s)
I20260812 16:40:51.287024 31754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
I20260812 16:40:51.287283 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 31754
I20260812 16:40:51.287590 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.194:44263
--local_ip_for_outbound_sockets=127.0.253.194
--tserver_master_addrs=127.0.253.254:39981
--webserver_port=41709
--webserver_interface=127.0.253.194
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:43519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:40:51.292085 31889 log.cc:824] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea: Log is configured to *not* fsync() on all Append() calls
I20260812 16:40:51.294420 31889 tablet_bootstrap.cc:492] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 16:40:51.294793 31889 tablet_bootstrap.cc:492] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea: Bootstrap complete.
I20260812 16:40:51.295877 31889 ts_tablet_manager.cc:1397] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.014s
I20260812 16:40:51.296979 31889 raft_consensus.cc:348] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d585036a34f74e2586fc4f8743561ea6" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44263 } } peers { permanent_uuid: "d400d8ef8d8c47128e143573073f6aa1" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 34789 } } peers { permanent_uuid: "0e7940e58fe64aa3a248e733e4e5ccea" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 39003 } }
I20260812 16:40:51.297245 31889 raft_consensus.cc:729] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e7940e58fe64aa3a248e733e4e5ccea, State: Initialized, Role: FOLLOWER
I20260812 16:40:51.297392 31889 consensus_queue.cc:260] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d585036a34f74e2586fc4f8743561ea6" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44263 } } peers { permanent_uuid: "d400d8ef8d8c47128e143573073f6aa1" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 34789 } } peers { permanent_uuid: "0e7940e58fe64aa3a248e733e4e5ccea" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 39003 } }
I20260812 16:40:51.297760 31889 ts_tablet_manager.cc:1428] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
W20260812 16:40:51.388263 32134 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:40:51.291224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.193:59377)
0812 16:40:51.291448 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:51.291493 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 16:40:51.292191 (+ 698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:40:51.388065 (+ 95874us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":130,"threads_started":1}
W20260812 16:40:51.388592 32131 heartbeater.cc:644] Failed to heartbeat to 127.0.253.254:39981 (0 consecutive failures): Network error: Failed to ping master at 127.0.253.254:39981: Client connection negotiation failed: client connection to 127.0.253.254:39981: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 16:40:51.389015 32134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:51.388898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.193:53173)
0812 16:40:51.388954 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:51.388996 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 16:40:51.389405 32134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:51.389326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.193:52625)
0812 16:40:51.389368 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:51.389390 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260812 16:40:51.389530 32131 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 16:40:51.396157 32130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:40:51.396358 32130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:40:51.396384 32130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:40:51.397539 32130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:40:51.397578 32130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:40:51.397604 32130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.194
I20260812 16:40:51.399118 32130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:43519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.194:44263
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.253.194
--webserver_port=41709
--tserver_master_addrs=127.0.253.254:39981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.194
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:40:51.399338 32130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:40:51.399525 32130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:40:51.404877 32147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:40:51.404884 32148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:40:51.405107 32130 server_base.cc:1034] running on GCE node
W20260812 16:40:51.404896 32150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:40:51.405373 32130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:40:51.405623 32130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:40:51.406790 32130 hybrid_clock.cc:648] HybridClock initialized: now 1786552851406777 us; error 24 us; skew 500 ppm
Aug 12 16:40:51 dist-test-slave-w4v5 krb5kdc[29431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552851, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:40:51.409962 32130 init.cc:377] Logged in from keytab as kudu/127.0.253.194@KRBTEST.COM (short username kudu)
I20260812 16:40:51.410355 32130 webserver.cc:460] Webserver started at http://127.0.253.194:41709/ using document root <none> and password file <none>
I20260812 16:40:51.410561 32130 fs_manager.cc:359] Metadata directory not provided
I20260812 16:40:51.410605 32130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:40:51.412135 32130 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:40:51.412725 32163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:40:51.412910 32130 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:40:51.413007 32130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/wal
uuid: "d585036a34f74e2586fc4f8743561ea6"
format_stamp: "Formatted at 2026-08-12 16:40:09 on dist-test-slave-w4v5"
server_key: "632f75a515053f5d72a5e61f9a2da7bd"
server_key_iv: "3f3c2fe50fa19c104b8674e9b09c1ea8"
server_key_version: "encryptionkey@0"
I20260812 16:40:51.413352 32130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:40:51.433256 32130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:40:51.435787 32130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:40:51.435947 32130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:40:51.436209 32130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:40:51.436558 32130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:40:51.436612 32130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:40:51.436637 32130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:40:51.436650 32130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:40:51.442981 32130 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.194:44263
I20260812 16:40:51.443022 32417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.194:44263 every 8 connection(s)
I20260812 16:40:51.443495 32130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
I20260812 16:40:51.444870 32425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:51.444238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.194:47185)
0812 16:40:51.444410 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:51.444483 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":81,"threads_started":1}
W20260812 16:40:51.445107 32421 heartbeater.cc:644] Failed to heartbeat to 127.0.253.254:39981 (0 consecutive failures): Network error: Failed to ping master at 127.0.253.254:39981: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
I20260812 16:40:51.445407 32425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:51.445309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.194:42775)
0812 16:40:51.445363 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:51.445390 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 16:40:51.445662 32425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:51.445609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.194:52639)
0812 16:40:51.445630 (+ 21us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:51.445647 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260812 16:40:51.445766 32421 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 16:40:51.453249 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 32130
I20260812 16:40:51.453524 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.195:34789
--local_ip_for_outbound_sockets=127.0.253.195
--tserver_master_addrs=127.0.253.254:39981
--webserver_port=45275
--webserver_interface=127.0.253.195
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:43519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:40:51.540024 32428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:40:51.540246 32428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:40:51.540271 32428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:40:51.541577 32428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:40:51.541623 32428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:40:51.541651 32428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.195
I20260812 16:40:51.543160 32428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:43519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.195:34789
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.253.195
--webserver_port=45275
--tserver_master_addrs=127.0.253.254:39981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.195
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:40:51.543426 32428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:40:51.543668 32428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:40:51.548949 32441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:40:51.548949 32448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:40:51.549037 32443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:40:51.549274 32428 server_base.cc:1034] running on GCE node
I20260812 16:40:51.549532 32428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:40:51.549813 32428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:40:51.550977 32428 hybrid_clock.cc:648] HybridClock initialized: now 1786552851550958 us; error 31 us; skew 500 ppm
Aug 12 16:40:51 dist-test-slave-w4v5 krb5kdc[29431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552851, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:40:51.553503 32428 init.cc:377] Logged in from keytab as kudu/127.0.253.195@KRBTEST.COM (short username kudu)
I20260812 16:40:51.554016 32428 webserver.cc:460] Webserver started at http://127.0.253.195:45275/ using document root <none> and password file <none>
I20260812 16:40:51.554270 32428 fs_manager.cc:359] Metadata directory not provided
I20260812 16:40:51.554345 32428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:40:51.555830 32428 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 16:40:51.556468 32463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:40:51.556658 32428 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 16:40:51.556787 32428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/wal
uuid: "d400d8ef8d8c47128e143573073f6aa1"
format_stamp: "Formatted at 2026-08-12 16:40:09 on dist-test-slave-w4v5"
server_key: "90db868f2b1bd673f0d16f3b6c1c461c"
server_key_iv: "f2a504d32ec34742b4894c1bf1554f0c"
server_key_version: "encryptionkey@0"
I20260812 16:40:51.557154 32428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:40:51.585693 32428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:40:51.588279 32428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:40:51.588485 32428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:40:51.588774 32428 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:40:51.589129 32428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:40:51.589196 32428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:40:51.589232 32428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:40:51.589254 32428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:40:51.595384 32428 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.195:34789
I20260812 16:40:51.595479 32710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.195:34789 every 8 connection(s)
I20260812 16:40:51.595870 32428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
I20260812 16:40:51.597018 32718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:51.596608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.195:40629)
0812 16:40:51.596801 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:51.596874 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":82,"threads_started":1}
W20260812 16:40:51.597513 32713 heartbeater.cc:644] Failed to heartbeat to 127.0.253.254:39981 (0 consecutive failures): Network error: Failed to ping master at 127.0.253.254:39981: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
I20260812 16:40:51.597854 32718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:51.597742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.195:43055)
0812 16:40:51.597800 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:51.597832 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 16:40:51.598229 32718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:51.598147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.195:39311)
0812 16:40:51.598189 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:51.598213 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260812 16:40:51.598367 32713 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 16:40:51.600219 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 32428
I20260812 16:40:52.390193 32724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:52.389894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.193:41071)
0812 16:40:52.390070 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:52.390114 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":72,"threads_started":1}
I20260812 16:40:52.446374 32728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:52.446128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.194:44969)
0812 16:40:52.446275 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:52.446316 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":68,"threads_started":1}
I20260812 16:40:52.598937 32730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:52.598704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.195:33711)
0812 16:40:52.598849 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:52.598883 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":90,"threads_started":1}
I20260812 16:40:52.941907 32733 raft_consensus.cc:482] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:40:52.942077 32733 raft_consensus.cc:504] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d585036a34f74e2586fc4f8743561ea6" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44263 } } peers { permanent_uuid: "d400d8ef8d8c47128e143573073f6aa1" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 34789 } } peers { permanent_uuid: "0e7940e58fe64aa3a248e733e4e5ccea" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 39003 } }
I20260812 16:40:52.942459 32733 leader_election.cc:288] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d585036a34f74e2586fc4f8743561ea6 (127.0.253.194:44263), d400d8ef8d8c47128e143573073f6aa1 (127.0.253.195:34789)
Aug 12 16:40:52 dist-test-slave-w4v5 krb5kdc[29431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552851, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for kudu/127.0.253.194@KRBTEST.COM
Aug 12 16:40:52 dist-test-slave-w4v5 krb5kdc[29431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552851, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for kudu/127.0.253.195@KRBTEST.COM
I20260812 16:40:52.953559 32736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:52.942753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:55363 (local address 127.0.253.194:44263)
0812 16:40:52.942912 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 16:40:52.942926 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 16:40:52.942968 (+ 42us) server_negotiation.cc:407] Connection header received
0812 16:40:52.943679 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:40:52.943684 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:40:52.943746 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:40:52.943920 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:40:52.944864 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:52.945769 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:52.946922 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:52.947376 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:52.947489 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:52.950132 (+ 2643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:40:52.950141 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:40:52.950142 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:40:52.950152 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:40:52.951850 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:40:52.952303 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:40:52.952306 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:40:52.952307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:40:52.952405 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:40:52.952664 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:40:52.952665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:40:52.952666 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:40:52.952913 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:40:52.953002 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 16:40:52.953323 (+ 321us) server_negotiation.cc:299] Negotiation successful
0812 16:40:52.953451 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":51,"threads_started":1}
I20260812 16:40:52.953548 32734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:52.942635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:44263 (local address 127.0.253.193:55363)
0812 16:40:52.942822 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:52.942835 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:40:52.942894 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:40:52.943952 (+ 1058us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:40:52.943955 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:40:52.944442 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:40:52.944726 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:40:52.944734 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:40:52.945901 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:40:52.945902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:40:52.946821 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:40:52.946824 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:40:52.947468 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:40:52.947469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:40:52.947598 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:52.948360 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:40:52.948369 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:40:52.949960 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:40:52.951954 (+ 1994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:40:52.951961 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:40:52.951964 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:40:52.952194 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:40:52.952483 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:40:52.952484 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:40:52.952484 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:40:52.952593 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:40:52.952994 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:40:52.953002 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:40:52.953139 (+ 137us) client_negotiation.cc:769] Sending connection context
0812 16:40:52.953264 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 16:40:52.953435 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":37,"thread_start_us":145,"threads_started":1}
I20260812 16:40:52.953677 32737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:52.942905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.195:34789 (local address 127.0.253.193:38129)
0812 16:40:52.943003 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:52.943015 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:40:52.943060 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:40:52.944055 (+ 995us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:40:52.944055 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:40:52.944459 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:40:52.944726 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:40:52.944734 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:40:52.945866 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:40:52.945868 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:40:52.946820 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:40:52.946823 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:40:52.947500 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:40:52.947501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:40:52.947603 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:52.948412 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:40:52.948417 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:40:52.950260 (+ 1843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:40:52.952356 (+ 2096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:40:52.952357 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:40:52.952357 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:40:52.952537 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:40:52.952807 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:40:52.952808 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:40:52.952808 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:40:52.952874 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:40:52.953266 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:40:52.953267 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:40:52.953312 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 16:40:52.953400 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 16:40:52.953520 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"mutex_wait_us":50,"thread_start_us":52,"threads_started":1}
I20260812 16:40:52.954360 32740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:52.943067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:38129 (local address 127.0.253.195:34789)
0812 16:40:52.943245 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 16:40:52.943255 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:40:52.943290 (+ 35us) server_negotiation.cc:407] Connection header received
0812 16:40:52.943865 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:40:52.943868 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:40:52.943918 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:40:52.944046 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:40:52.944875 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:52.945747 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:52.946914 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:40:52.947388 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:40:52.947514 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:40:52.950380 (+ 2866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:40:52.950384 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:40:52.950384 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:40:52.950395 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:40:52.952246 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:40:52.952629 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:40:52.952632 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:40:52.952633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:40:52.952727 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:40:52.952940 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:40:52.952941 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:40:52.952941 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:40:52.953181 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:40:52.953271 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 16:40:52.953465 (+ 194us) server_negotiation.cc:299] Negotiation successful
0812 16:40:52.954265 (+ 800us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":66,"threads_started":1}
W20260812 16:40:52.954430 31883 leader_election.cc:341] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d585036a34f74e2586fc4f8743561ea6 (127.0.253.194:44263): Not found: Tablet not found: b84942e43af64d09a6fbffb23c9ba263
W20260812 16:40:52.955003 31883 leader_election.cc:341] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d400d8ef8d8c47128e143573073f6aa1 (127.0.253.195:34789): Not found: Tablet not found: b84942e43af64d09a6fbffb23c9ba263
I20260812 16:40:52.955060 31883 leader_election.cc:302] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e7940e58fe64aa3a248e733e4e5ccea; no voters: d400d8ef8d8c47128e143573073f6aa1, d585036a34f74e2586fc4f8743561ea6
I20260812 16:40:52.955266 32733 raft_consensus.cc:2726] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 16:40:53.390856 32734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:53.390710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.193:59843)
0812 16:40:53.390771 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:53.390813 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 16:40:53.447155 32742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:53.446856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.194:37805)
0812 16:40:53.446992 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:53.447032 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":66,"threads_started":1}
I20260812 16:40:53.599637 32743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:53.599389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.195:41033)
0812 16:40:53.599522 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:53.599561 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":78,"threads_started":1}
I20260812 16:40:54.391708 32744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:54.391440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.193:42663)
0812 16:40:54.391610 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:54.391655 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":72,"threads_started":1}
I20260812 16:40:54.447878 32745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:54.447644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.194:58069)
0812 16:40:54.447793 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:54.447833 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":87,"threads_started":1}
I20260812 16:40:54.536453 32747 raft_consensus.cc:482] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:40:54.536608 32747 raft_consensus.cc:504] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d585036a34f74e2586fc4f8743561ea6" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44263 } } peers { permanent_uuid: "d400d8ef8d8c47128e143573073f6aa1" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 34789 } } peers { permanent_uuid: "0e7940e58fe64aa3a248e733e4e5ccea" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 39003 } }
I20260812 16:40:54.536862 32747 leader_election.cc:288] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d585036a34f74e2586fc4f8743561ea6 (127.0.253.194:44263), d400d8ef8d8c47128e143573073f6aa1 (127.0.253.195:34789)
W20260812 16:40:54.537331 31883 leader_election.cc:341] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d585036a34f74e2586fc4f8743561ea6 (127.0.253.194:44263): Not found: Tablet not found: b84942e43af64d09a6fbffb23c9ba263
W20260812 16:40:54.537423 31883 leader_election.cc:341] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d400d8ef8d8c47128e143573073f6aa1 (127.0.253.195:34789): Not found: Tablet not found: b84942e43af64d09a6fbffb23c9ba263
I20260812 16:40:54.537464 31883 leader_election.cc:302] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e7940e58fe64aa3a248e733e4e5ccea; no voters: d400d8ef8d8c47128e143573073f6aa1, d585036a34f74e2586fc4f8743561ea6
I20260812 16:40:54.537541 32747 raft_consensus.cc:2726] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 16:40:54.600333 32748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:54.600113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.195:39357)
0812 16:40:54.600253 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:54.600292 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":70,"threads_started":1}
I20260812 16:40:55.392490 32750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:55.392261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.193:46629)
0812 16:40:55.392395 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:55.392441 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":68,"threads_started":1}
I20260812 16:40:55.448634 32757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:55.448381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.194:43175)
0812 16:40:55.448525 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:55.448562 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":84,"threads_started":1}
I20260812 16:40:55.601116 32758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:55.600899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.195:38185)
0812 16:40:55.601031 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:55.601073 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":66,"threads_started":1}
I20260812 16:40:56.393468 32759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:56.393199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.193:46523)
0812 16:40:56.393377 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:56.393419 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":91,"threads_started":1}
I20260812 16:40:56.449378 32760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:56.449158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.194:43115)
0812 16:40:56.449298 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:56.449333 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":83,"threads_started":1}
I20260812 16:40:56.601884 32761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:56.601624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.195:59575)
0812 16:40:56.601791 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:56.601836 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":87,"threads_started":1}
I20260812 16:40:56.674059 32763 raft_consensus.cc:482] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:40:56.674157 32763 raft_consensus.cc:504] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d585036a34f74e2586fc4f8743561ea6" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44263 } } peers { permanent_uuid: "d400d8ef8d8c47128e143573073f6aa1" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 34789 } } peers { permanent_uuid: "0e7940e58fe64aa3a248e733e4e5ccea" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 39003 } }
I20260812 16:40:56.674372 32763 leader_election.cc:288] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d585036a34f74e2586fc4f8743561ea6 (127.0.253.194:44263), d400d8ef8d8c47128e143573073f6aa1 (127.0.253.195:34789)
W20260812 16:40:56.674821 31883 leader_election.cc:341] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d585036a34f74e2586fc4f8743561ea6 (127.0.253.194:44263): Not found: Tablet not found: b84942e43af64d09a6fbffb23c9ba263
W20260812 16:40:56.674901 31883 leader_election.cc:341] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d400d8ef8d8c47128e143573073f6aa1 (127.0.253.195:34789): Not found: Tablet not found: b84942e43af64d09a6fbffb23c9ba263
I20260812 16:40:56.674918 31883 leader_election.cc:302] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e7940e58fe64aa3a248e733e4e5ccea; no voters: d400d8ef8d8c47128e143573073f6aa1, d585036a34f74e2586fc4f8743561ea6
I20260812 16:40:56.674988 32763 raft_consensus.cc:2726] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 16:40:57.394260 32764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:57.394009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.193:34009)
0812 16:40:57.394169 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:57.394212 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":77,"threads_started":1}
I20260812 16:40:57.450088 32765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:57.449853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.194:57277)
0812 16:40:57.450003 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:57.450041 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":82,"threads_started":1}
I20260812 16:40:57.602665 32766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:57.602419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.195:49233)
0812 16:40:57.602574 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:57.602616 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":64,"threads_started":1}
I20260812 16:40:58.395305 300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:58.394939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.193:59183)
0812 16:40:58.395200 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:58.395253 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":93,"threads_started":1}
I20260812 16:40:58.450944 301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:58.450650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.194:60129)
0812 16:40:58.450844 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:58.450896 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":91,"threads_started":1}
I20260812 16:40:58.603502 302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:58.603252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.195:49669)
0812 16:40:58.603410 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:58.603456 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":83,"threads_started":1}
I20260812 16:40:59.396365 303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:59.396063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.193:41727)
0812 16:40:59.396268 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:59.396315 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":114,"threads_started":1}
I20260812 16:40:59.451824 304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:59.451586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.194:48299)
0812 16:40:59.451745 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:59.451780 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":97,"threads_started":1}
I20260812 16:40:59.604390 307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:40:59.604118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.195:48467)
0812 16:40:59.604300 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 16:40:59.604343 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":84,"threads_started":1}
I20260812 16:40:59.736235 309 raft_consensus.cc:482] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:40:59.736342 309 raft_consensus.cc:504] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d585036a34f74e2586fc4f8743561ea6" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44263 } } peers { permanent_uuid: "d400d8ef8d8c47128e143573073f6aa1" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 34789 } } peers { permanent_uuid: "0e7940e58fe64aa3a248e733e4e5ccea" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 39003 } }
I20260812 16:40:59.736479 309 leader_election.cc:288] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d585036a34f74e2586fc4f8743561ea6 (127.0.253.194:44263), d400d8ef8d8c47128e143573073f6aa1 (127.0.253.195:34789)
W20260812 16:40:59.737010 31883 leader_election.cc:341] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d400d8ef8d8c47128e143573073f6aa1 (127.0.253.195:34789): Not found: Tablet not found: b84942e43af64d09a6fbffb23c9ba263
W20260812 16:40:59.737085 31883 leader_election.cc:341] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d585036a34f74e2586fc4f8743561ea6 (127.0.253.194:44263): Not found: Tablet not found: b84942e43af64d09a6fbffb23c9ba263
I20260812 16:40:59.737102 31883 leader_election.cc:302] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e7940e58fe64aa3a248e733e4e5ccea; no voters: d400d8ef8d8c47128e143573073f6aa1, d585036a34f74e2586fc4f8743561ea6
I20260812 16:40:59.737226 309 raft_consensus.cc:2726] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 16:41:00.397114 311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:00.396876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.193:35019)
0812 16:41:00.397018 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:00.397061 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":73,"threads_started":1}
I20260812 16:41:00.452627 315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:00.452416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.194:36421)
0812 16:41:00.452541 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:00.452575 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":67,"threads_started":1}
I20260812 16:41:00.605082 318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:00.604861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.195:33661)
0812 16:41:00.605002 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:00.605040 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":66,"threads_started":1}
I20260812 16:41:01.398059 323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:01.397814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.193:55433)
0812 16:41:01.397961 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:01.398009 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":70,"threads_started":1}
I20260812 16:41:01.453454 327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:01.453178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.194:58611)
0812 16:41:01.453336 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:01.453385 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":83,"threads_started":1}
I20260812 16:41:01.605916 328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:01.605664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.195:57773)
0812 16:41:01.605829 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:01.605869 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":75,"threads_started":1}
I20260812 16:41:02.398896 329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:02.398646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.193:36577)
0812 16:41:02.398802 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:02.398847 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":68,"threads_started":1}
I20260812 16:41:02.454448 330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:02.454178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.194:46669)
0812 16:41:02.454357 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:02.454398 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":107,"threads_started":1}
I20260812 16:41:02.606801 335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:02.606446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.195:60425)
0812 16:41:02.606718 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:02.606755 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":209,"threads_started":1}
I20260812 16:41:03.118703 336 raft_consensus.cc:482] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:41:03.118806 336 raft_consensus.cc:504] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d585036a34f74e2586fc4f8743561ea6" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44263 } } peers { permanent_uuid: "d400d8ef8d8c47128e143573073f6aa1" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 34789 } } peers { permanent_uuid: "0e7940e58fe64aa3a248e733e4e5ccea" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 39003 } }
I20260812 16:41:03.119048 336 leader_election.cc:288] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d585036a34f74e2586fc4f8743561ea6 (127.0.253.194:44263), d400d8ef8d8c47128e143573073f6aa1 (127.0.253.195:34789)
W20260812 16:41:03.119514 31883 leader_election.cc:341] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d400d8ef8d8c47128e143573073f6aa1 (127.0.253.195:34789): Not found: Tablet not found: b84942e43af64d09a6fbffb23c9ba263
W20260812 16:41:03.119602 31883 leader_election.cc:341] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d585036a34f74e2586fc4f8743561ea6 (127.0.253.194:44263): Not found: Tablet not found: b84942e43af64d09a6fbffb23c9ba263
I20260812 16:41:03.119622 31883 leader_election.cc:302] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e7940e58fe64aa3a248e733e4e5ccea; no voters: d400d8ef8d8c47128e143573073f6aa1, d585036a34f74e2586fc4f8743561ea6
I20260812 16:41:03.119704 336 raft_consensus.cc:2726] T b84942e43af64d09a6fbffb23c9ba263 P 0e7940e58fe64aa3a248e733e4e5ccea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 16:41:03.399776 337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:03.399518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.193:36955)
0812 16:41:03.399674 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:03.399723 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":79,"threads_started":1}
I20260812 16:41:03.455225 344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:03.454900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.194:51007)
0812 16:41:03.455058 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:03.455155 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":95,"threads_started":1}
I20260812 16:41:03.607581 346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:03.607367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.195:53161)
0812 16:41:03.607497 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:03.607534 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":67,"threads_started":1}
I20260812 16:41:04.400558 352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:04.400308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.193:44131)
0812 16:41:04.400455 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:04.400506 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":66,"threads_started":1}
I20260812 16:41:04.455981 354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:04.455741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.194:57475)
0812 16:41:04.455892 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:04.455933 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":72,"threads_started":1}
I20260812 16:41:04.608304 355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:04.608066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.195:50553)
0812 16:41:04.608217 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:04.608255 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":86,"threads_started":1}
I20260812 16:41:05.401269 356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:05.401049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.193:52689)
0812 16:41:05.401178 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:05.401216 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":71,"threads_started":1}
I20260812 16:41:05.456700 357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:05.456463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.194:48467)
0812 16:41:05.456610 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:05.456651 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":78,"threads_started":1}
I20260812 16:41:05.609036 359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:05.608815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.195:45887)
0812 16:41:05.608947 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:05.608986 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":66,"threads_started":1}
I20260812 16:41:06.402171 361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:06.401932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.193:54461)
0812 16:41:06.402078 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:06.402124 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":66,"threads_started":1}
I20260812 16:41:06.457417 364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:06.457205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.194:55869)
0812 16:41:06.457330 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:06.457369 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":66,"threads_started":1}
I20260812 16:41:06.601796 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 31754
I20260812 16:41:06.612694 365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:06.609768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:39981 (local address 127.0.253.195:60613)
0812 16:41:06.612554 (+ 2786us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:06.612614 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.254:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2760,"thread_start_us":1438,"threads_started":1}
I20260812 16:41:06.617312 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 32130
I20260812 16:41:06.635237 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 32428
2026-08-12T16:41:06Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58899 ms)
[----------] 30 tests from SecurityITest (247341 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:41:06 dist-test-slave-w4v5 krb5kdc[369](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:41:06 dist-test-slave-w4v5 krb5kdc[369](info): set up 2 sockets
Aug 12 16:41:06 dist-test-slave-w4v5 krb5kdc[369](info): commencing operation
krb5kdc: starting...
W20260812 16:41:08.012457 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.347s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:41:08 dist-test-slave-w4v5 krb5kdc[369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:41:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:41:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
I20260812 16:41:08.138368 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:36741
--webserver_interface=127.0.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:33947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:36741
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:41:08.222383 400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:08.222602 400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:08.222625 400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:08.223743 400 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:41:08.223779 400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:08.223793 400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:08.223806 400 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:41:08.223819 400 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:41:08.225226 400 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:33947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:36741
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:36741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:08.225423 400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:08.225586 400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:08.230894 409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:08.230865 410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:08.230885 400 server_base.cc:1034] running on GCE node
W20260812 16:41:08.230937 417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:08.231352 400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:08.231596 400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:08.232777 400 hybrid_clock.cc:648] HybridClock initialized: now 1786552868232754 us; error 37 us; skew 500 ppm
Aug 12 16:41:08 dist-test-slave-w4v5 krb5kdc[369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552868, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:41:08.235505 400 init.cc:377] Logged in from keytab as kudu/127.0.253.254@KRBTEST.COM (short username kudu)
I20260812 16:41:08.235945 400 webserver.cc:460] Webserver started at http://127.0.253.254:43583/ using document root <none> and password file <none>
I20260812 16:41:08.236207 400 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:08.236279 400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:08.236455 400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:08.237593 400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "edea5da71d5c45ec8f448260af5138c1"
format_stamp: "Formatted at 2026-08-12 16:41:08 on dist-test-slave-w4v5"
server_key: "95ab6d2a2fe677abc5aaaf7a4168beed"
server_key_iv: "58cf16f13ca49f469c3db7017e77f487"
server_key_version: "encryptionkey@0"
I20260812 16:41:08.237965 400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "edea5da71d5c45ec8f448260af5138c1"
format_stamp: "Formatted at 2026-08-12 16:41:08 on dist-test-slave-w4v5"
server_key: "95ab6d2a2fe677abc5aaaf7a4168beed"
server_key_iv: "58cf16f13ca49f469c3db7017e77f487"
server_key_version: "encryptionkey@0"
I20260812 16:41:08.239373 400 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260812 16:41:08.240094 430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:08.240267 400 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:41:08.240371 400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "edea5da71d5c45ec8f448260af5138c1"
format_stamp: "Formatted at 2026-08-12 16:41:08 on dist-test-slave-w4v5"
server_key: "95ab6d2a2fe677abc5aaaf7a4168beed"
server_key_iv: "58cf16f13ca49f469c3db7017e77f487"
server_key_version: "encryptionkey@0"
I20260812 16:41:08.240459 400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:08.252931 400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:08.255148 400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:08.255323 400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:08.259537 400 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:36741
I20260812 16:41:08.259575 550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:36741 every 8 connection(s)
I20260812 16:41:08.259963 400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:41:08.260934 554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:08.263352 554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edea5da71d5c45ec8f448260af5138c1: Bootstrap starting.
I20260812 16:41:08.263981 554 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P edea5da71d5c45ec8f448260af5138c1: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:08.264008 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 400
I20260812 16:41:08.264156 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:41:08.264307 554 log.cc:824] T 00000000000000000000000000000000 P edea5da71d5c45ec8f448260af5138c1: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:08.264393 1015 external_mini_cluster.cc:1428] Setting key bf81470005cc5d81ef8085506b4294c7
I20260812 16:41:08.265218 554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edea5da71d5c45ec8f448260af5138c1: No bootstrap required, opened a new log
I20260812 16:41:08.267891 554 raft_consensus.cc:348] T 00000000000000000000000000000000 P edea5da71d5c45ec8f448260af5138c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edea5da71d5c45ec8f448260af5138c1" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36741 } }
I20260812 16:41:08.268061 554 raft_consensus.cc:374] T 00000000000000000000000000000000 P edea5da71d5c45ec8f448260af5138c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:08.268095 554 raft_consensus.cc:729] T 00000000000000000000000000000000 P edea5da71d5c45ec8f448260af5138c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edea5da71d5c45ec8f448260af5138c1, State: Initialized, Role: FOLLOWER
I20260812 16:41:08.268229 554 consensus_queue.cc:260] T 00000000000000000000000000000000 P edea5da71d5c45ec8f448260af5138c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edea5da71d5c45ec8f448260af5138c1" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36741 } }
I20260812 16:41:08.268322 554 raft_consensus.cc:388] T 00000000000000000000000000000000 P edea5da71d5c45ec8f448260af5138c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:41:08.268355 554 raft_consensus.cc:482] T 00000000000000000000000000000000 P edea5da71d5c45ec8f448260af5138c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:41:08.268409 554 raft_consensus.cc:3037] T 00000000000000000000000000000000 P edea5da71d5c45ec8f448260af5138c1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:08.269042 554 raft_consensus.cc:504] T 00000000000000000000000000000000 P edea5da71d5c45ec8f448260af5138c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edea5da71d5c45ec8f448260af5138c1" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36741 } }
I20260812 16:41:08.269152 554 leader_election.cc:302] T 00000000000000000000000000000000 P edea5da71d5c45ec8f448260af5138c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: edea5da71d5c45ec8f448260af5138c1; no voters:
I20260812 16:41:08.269320 554 leader_election.cc:288] T 00000000000000000000000000000000 P edea5da71d5c45ec8f448260af5138c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:41:08.269400 564 raft_consensus.cc:2781] T 00000000000000000000000000000000 P edea5da71d5c45ec8f448260af5138c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:41:08.269559 554 sys_catalog.cc:564] T 00000000000000000000000000000000 P edea5da71d5c45ec8f448260af5138c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:41:08.269596 564 raft_consensus.cc:686] T 00000000000000000000000000000000 P edea5da71d5c45ec8f448260af5138c1 [term 1 LEADER]: Becoming Leader. State: Replica: edea5da71d5c45ec8f448260af5138c1, State: Running, Role: LEADER
I20260812 16:41:08.269838 564 consensus_queue.cc:237] T 00000000000000000000000000000000 P edea5da71d5c45ec8f448260af5138c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edea5da71d5c45ec8f448260af5138c1" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36741 } }
I20260812 16:41:08.270462 565 sys_catalog.cc:455] T 00000000000000000000000000000000 P edea5da71d5c45ec8f448260af5138c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "edea5da71d5c45ec8f448260af5138c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edea5da71d5c45ec8f448260af5138c1" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36741 } } }
I20260812 16:41:08.270583 565 sys_catalog.cc:458] T 00000000000000000000000000000000 P edea5da71d5c45ec8f448260af5138c1 [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:08.270938 567 sys_catalog.cc:455] T 00000000000000000000000000000000 P edea5da71d5c45ec8f448260af5138c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader edea5da71d5c45ec8f448260af5138c1. Latest consensus state: current_term: 1 leader_uuid: "edea5da71d5c45ec8f448260af5138c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edea5da71d5c45ec8f448260af5138c1" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36741 } } }
I20260812 16:41:08.271170 567 sys_catalog.cc:458] T 00000000000000000000000000000000 P edea5da71d5c45ec8f448260af5138c1 [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:08.271160 576 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:41:08.271826 576 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 16:41:08 dist-test-slave-w4v5 krb5kdc[369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:41:08.273442 576 catalog_manager.cc:1347] Generated new cluster ID: f282ac6ef98f44ccbb61e0539159d8cb
I20260812 16:41:08.273497 576 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:41:08.276705 560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:08.264895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43320 (local address 127.0.253.254:36741)
0812 16:41:08.265084 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:08.265094 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:08.265129 (+ 35us) server_negotiation.cc:407] Connection header received
0812 16:41:08.265689 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:08.265707 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:08.265775 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:08.265932 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:08.266557 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:08.268134 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:08.269013 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:08.269642 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:08.270135 (+ 493us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:08.272744 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:08.272748 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:08.272748 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:08.272761 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:08.274924 (+ 2163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:08.275403 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:08.275407 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:08.275407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:08.275519 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:08.275746 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:08.275747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:08.275747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:08.276065 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:08.276244 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:08.276339 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 16:41:08.276482 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":80,"threads_started":1}
I20260812 16:41:08.284408 576 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:41:08.284965 576 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:41:08.292966 576 catalog_manager.cc:5868] T 00000000000000000000000000000000 P edea5da71d5c45ec8f448260af5138c1: Generated new TSK 0
I20260812 16:41:08.293184 576 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab.
Entry for principal kudu/127.0.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab.
I20260812 16:41:08.326303 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.193:0
--local_ip_for_outbound_sockets=127.0.253.193
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:36741
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:33947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:41:08.416342 610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:08.416561 610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:08.416587 610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:08.417775 610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:08.417819 610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:08.417846 610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.193
I20260812 16:41:08.419363 610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:33947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:36741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.193
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:08.419620 610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:08.419871 610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:08.425086 619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:08.425107 623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:08.425101 616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:08.425335 610 server_base.cc:1034] running on GCE node
I20260812 16:41:08.425544 610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:08.425850 610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:08.427019 610 hybrid_clock.cc:648] HybridClock initialized: now 1786552868427006 us; error 44 us; skew 500 ppm
Aug 12 16:41:08 dist-test-slave-w4v5 krb5kdc[369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552868, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:41:08.429718 610 init.cc:377] Logged in from keytab as kudu/127.0.253.193@KRBTEST.COM (short username kudu)
I20260812 16:41:08.430163 610 webserver.cc:460] Webserver started at http://127.0.253.193:44349/ using document root <none> and password file <none>
I20260812 16:41:08.430411 610 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:08.430480 610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:08.430653 610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:08.431810 610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-0/data/instance:
uuid: "0ed88041ad3048ad8089fe493d305306"
format_stamp: "Formatted at 2026-08-12 16:41:08 on dist-test-slave-w4v5"
server_key: "e094d5b9cb2ba1d5797c450ba0e38aaf"
server_key_iv: "43074bd9e4c82f948d8814275d04bb96"
server_key_version: "encryptionkey@0"
I20260812 16:41:08.432179 610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance:
uuid: "0ed88041ad3048ad8089fe493d305306"
format_stamp: "Formatted at 2026-08-12 16:41:08 on dist-test-slave-w4v5"
server_key: "e094d5b9cb2ba1d5797c450ba0e38aaf"
server_key_iv: "43074bd9e4c82f948d8814275d04bb96"
server_key_version: "encryptionkey@0"
I20260812 16:41:08.433626 610 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:41:08.434474 635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:08.434772 610 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:41:08.434873 610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-0/wal
uuid: "0ed88041ad3048ad8089fe493d305306"
format_stamp: "Formatted at 2026-08-12 16:41:08 on dist-test-slave-w4v5"
server_key: "e094d5b9cb2ba1d5797c450ba0e38aaf"
server_key_iv: "43074bd9e4c82f948d8814275d04bb96"
server_key_version: "encryptionkey@0"
I20260812 16:41:08.434959 610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:08.448904 610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:08.451499 610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:08.451682 610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:08.451960 610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:41:08.452296 610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:41:08.452351 610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:08.452399 610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:41:08.452425 610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:08.458711 610 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.193:36977
I20260812 16:41:08.458766 872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.193:36977 every 8 connection(s)
I20260812 16:41:08.459115 610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
I20260812 16:41:08.463037 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 610
I20260812 16:41:08.463179 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance
I20260812 16:41:08.463400 1015 external_mini_cluster.cc:1428] Setting key cabeff93e1018bff53566f218ac9a085
Aug 12 16:41:08 dist-test-slave-w4v5 krb5kdc[369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552868, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:41:08.469854 560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:08.459892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:36281 (local address 127.0.253.254:36741)
0812 16:41:08.459964 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:08.459965 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:08.460171 (+ 206us) server_negotiation.cc:407] Connection header received
0812 16:41:08.460722 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:08.460723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:08.460735 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:08.460812 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:08.461837 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:08.462375 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:08.463793 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:08.464037 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:08.464127 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:08.466597 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:08.466599 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:08.466599 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:08.466607 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:08.468181 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:08.468741 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:08.468742 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:08.468743 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:08.468818 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:08.469128 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:08.469129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:08.469129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:08.469243 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:08.469300 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:08.469662 (+ 362us) server_negotiation.cc:299] Negotiation successful
0812 16:41:08.469773 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 16:41:08.470113 877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:08.459845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36741 (local address 127.0.253.193:36281)
0812 16:41:08.460017 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:08.460060 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:08.460598 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:08.460903 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:08.460907 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:08.461481 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:41:08.461717 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:08.461723 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:08.462460 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:08.462461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:08.463696 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:08.463699 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:08.464140 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:08.464142 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:08.464289 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:08.464852 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:41:08.464861 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:41:08.466425 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:41:08.468271 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:08.468280 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:08.468283 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:08.468647 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:08.468897 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:08.468898 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:08.468898 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:08.469056 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:08.469313 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:41:08.469321 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:41:08.469502 (+ 181us) client_negotiation.cc:769] Sending connection context
0812 16:41:08.469634 (+ 132us) client_negotiation.cc:240] Negotiation successful
0812 16:41:08.469837 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":92,"threads_started":1}
I20260812 16:41:08.470796 874 heartbeater.cc:347] Connected to a master server at 127.0.253.254:36741
I20260812 16:41:08.470956 874 heartbeater.cc:464] Registering TS with master...
I20260812 16:41:08.471199 874 heartbeater.cc:511] Master 127.0.253.254:36741 requested a full tablet report, sending...
I20260812 16:41:08.471824 468 ts_manager.cc:194] Registered new tserver with Master: 0ed88041ad3048ad8089fe493d305306 (127.0.253.193:36977)
I20260812 16:41:08.472474 468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.193@KRBTEST.COM'} at 127.0.253.193:36281
WARNING: no policy specified for kudu/127.0.253.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab.
Entry for principal kudu/127.0.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab.
I20260812 16:41:08.491622 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.194:0
--local_ip_for_outbound_sockets=127.0.253.194
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:36741
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:33947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:41:08.576524 880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:08.576764 880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:08.576802 880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:08.577979 880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:08.578044 880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:08.578090 880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.194
I20260812 16:41:08.579620 880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:33947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:36741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.880
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.194
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:08.579890 880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:08.580137 880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:08.585631 890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:08.585676 880 server_base.cc:1034] running on GCE node
W20260812 16:41:08.585623 894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:08.585606 897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:08.586077 880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:08.586333 880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:08.587486 880 hybrid_clock.cc:648] HybridClock initialized: now 1786552868587467 us; error 36 us; skew 500 ppm
Aug 12 16:41:08 dist-test-slave-w4v5 krb5kdc[369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552868, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:41:08.590229 880 init.cc:377] Logged in from keytab as kudu/127.0.253.194@KRBTEST.COM (short username kudu)
I20260812 16:41:08.590706 880 webserver.cc:460] Webserver started at http://127.0.253.194:40751/ using document root <none> and password file <none>
I20260812 16:41:08.590956 880 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:08.591002 880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:08.591185 880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:08.592222 880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-1/data/instance:
uuid: "6283554482d8443eaf5f2ed2288c7701"
format_stamp: "Formatted at 2026-08-12 16:41:08 on dist-test-slave-w4v5"
server_key: "da66baa39f0acb66ddc1ff9e79a8401f"
server_key_iv: "fd394a5b92858d892a18bcba53a08926"
server_key_version: "encryptionkey@0"
I20260812 16:41:08.592588 880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance:
uuid: "6283554482d8443eaf5f2ed2288c7701"
format_stamp: "Formatted at 2026-08-12 16:41:08 on dist-test-slave-w4v5"
server_key: "da66baa39f0acb66ddc1ff9e79a8401f"
server_key_iv: "fd394a5b92858d892a18bcba53a08926"
server_key_version: "encryptionkey@0"
I20260812 16:41:08.593954 880 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:41:08.594735 914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:08.594925 880 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:41:08.595031 880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-1/wal
uuid: "6283554482d8443eaf5f2ed2288c7701"
format_stamp: "Formatted at 2026-08-12 16:41:08 on dist-test-slave-w4v5"
server_key: "da66baa39f0acb66ddc1ff9e79a8401f"
server_key_iv: "fd394a5b92858d892a18bcba53a08926"
server_key_version: "encryptionkey@0"
I20260812 16:41:08.595150 880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:08.608214 880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:08.610760 880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:08.610936 880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:08.611279 880 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:41:08.611624 880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:41:08.611660 880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:08.611683 880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:41:08.611732 880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:08.617647 880 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.194:41953
I20260812 16:41:08.617765 1189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.194:41953 every 8 connection(s)
I20260812 16:41:08.618076 880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
Aug 12 16:41:08 dist-test-slave-w4v5 krb5kdc[369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552868, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:41:08.627334 560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:08.618958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:51929 (local address 127.0.253.254:36741)
0812 16:41:08.619038 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:08.619040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:08.619229 (+ 189us) server_negotiation.cc:407] Connection header received
0812 16:41:08.619732 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:08.619733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:08.619745 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:08.619812 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:08.620773 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:08.621367 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:08.622431 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:08.622626 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:08.622696 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:08.624953 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:08.624954 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:08.624954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:08.624961 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:08.626129 (+ 1168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:08.626481 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:08.626481 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:08.626482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:08.626530 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:08.626741 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:08.626742 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:08.626742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:08.626843 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:08.626892 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:08.627189 (+ 297us) server_negotiation.cc:299] Negotiation successful
0812 16:41:08.627267 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 16:41:08.627339 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 880
I20260812 16:41:08.627454 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance
I20260812 16:41:08.627600 1194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:08.618872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36741 (local address 127.0.253.194:51929)
0812 16:41:08.619094 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:08.619138 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:08.619632 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:08.619866 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:08.619869 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:08.620443 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:41:08.620658 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:08.620664 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:08.621493 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:08.621495 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:08.622322 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:08.622325 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:08.622716 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:08.622717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:08.622819 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:08.623447 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:41:08.623456 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:41:08.624859 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:41:08.626199 (+ 1340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:08.626205 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:08.626207 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:08.626418 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:08.626587 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:08.626588 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:08.626588 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:08.626688 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:08.626896 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:41:08.626901 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:41:08.627011 (+ 110us) client_negotiation.cc:769] Sending connection context
0812 16:41:08.627158 (+ 147us) client_negotiation.cc:240] Negotiation successful
0812 16:41:08.627343 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":106,"threads_started":1}
I20260812 16:41:08.627707 1015 external_mini_cluster.cc:1428] Setting key f04c9089b520e14cf7ebd5b453826a35
I20260812 16:41:08.628271 1190 heartbeater.cc:347] Connected to a master server at 127.0.253.254:36741
I20260812 16:41:08.628383 1190 heartbeater.cc:464] Registering TS with master...
I20260812 16:41:08.628590 1190 heartbeater.cc:511] Master 127.0.253.254:36741 requested a full tablet report, sending...
I20260812 16:41:08.629012 465 ts_manager.cc:194] Registered new tserver with Master: 6283554482d8443eaf5f2ed2288c7701 (127.0.253.194:41953)
I20260812 16:41:08.629523 465 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.194@KRBTEST.COM'} at 127.0.253.194:51929
WARNING: no policy specified for kudu/127.0.253.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab.
Entry for principal kudu/127.0.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab.
I20260812 16:41:08.655668 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.195:0
--local_ip_for_outbound_sockets=127.0.253.195
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:36741
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:33947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:41:08.740442 1201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:08.740679 1201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:08.740720 1201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:08.741870 1201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:08.741941 1201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:08.741986 1201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.195
I20260812 16:41:08.743537 1201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:33947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:36741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.195
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:08.743808 1201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:08.744040 1201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:08.749441 1219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:08.749436 1216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:08.749527 1201 server_base.cc:1034] running on GCE node
W20260812 16:41:08.749432 1214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:08.749919 1201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:08.750183 1201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:08.751327 1201 hybrid_clock.cc:648] HybridClock initialized: now 1786552868751310 us; error 31 us; skew 500 ppm
Aug 12 16:41:08 dist-test-slave-w4v5 krb5kdc[369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552868, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:41:08.753909 1201 init.cc:377] Logged in from keytab as kudu/127.0.253.195@KRBTEST.COM (short username kudu)
I20260812 16:41:08.754381 1201 webserver.cc:460] Webserver started at http://127.0.253.195:42599/ using document root <none> and password file <none>
I20260812 16:41:08.754640 1201 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:08.754710 1201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:08.754861 1201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:08.756021 1201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-2/data/instance:
uuid: "5e96ebcc70da4cc3ab9f1ed3e6bc6bc6"
format_stamp: "Formatted at 2026-08-12 16:41:08 on dist-test-slave-w4v5"
server_key: "c4260b0ffdaa6cf1d919509e9d0faa07"
server_key_iv: "f117ad538ae697c2282d205607d02101"
server_key_version: "encryptionkey@0"
I20260812 16:41:08.756385 1201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance:
uuid: "5e96ebcc70da4cc3ab9f1ed3e6bc6bc6"
format_stamp: "Formatted at 2026-08-12 16:41:08 on dist-test-slave-w4v5"
server_key: "c4260b0ffdaa6cf1d919509e9d0faa07"
server_key_iv: "f117ad538ae697c2282d205607d02101"
server_key_version: "encryptionkey@0"
I20260812 16:41:08.759474 1201 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 16:41:08.760571 1236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:08.760777 1201 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:41:08.760885 1201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-2/wal
uuid: "5e96ebcc70da4cc3ab9f1ed3e6bc6bc6"
format_stamp: "Formatted at 2026-08-12 16:41:08 on dist-test-slave-w4v5"
server_key: "c4260b0ffdaa6cf1d919509e9d0faa07"
server_key_iv: "f117ad538ae697c2282d205607d02101"
server_key_version: "encryptionkey@0"
I20260812 16:41:08.760973 1201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:08.778684 1201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:08.781241 1201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:08.781366 1201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:08.781608 1201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:41:08.781960 1201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:41:08.781996 1201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:08.782020 1201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:41:08.782033 1201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:08.788126 1201 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.195:38873
I20260812 16:41:08.788173 1476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.195:38873 every 8 connection(s)
I20260812 16:41:08.788522 1201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
I20260812 16:41:08.791333 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 1201
I20260812 16:41:08.791435 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance
I20260812 16:41:08.791621 1015 external_mini_cluster.cc:1428] Setting key ee0c2125d78046dbf3337ab4b725802d
Aug 12 16:41:08 dist-test-slave-w4v5 krb5kdc[369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552868, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:41:08.798194 560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:08.789229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:47495 (local address 127.0.253.254:36741)
0812 16:41:08.789307 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:08.789309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:08.789538 (+ 229us) server_negotiation.cc:407] Connection header received
0812 16:41:08.790093 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:08.790095 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:08.790108 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:08.790167 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:08.791255 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:08.791929 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:08.793319 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:08.793508 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:08.793563 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:08.795661 (+ 2098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:08.795662 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:08.795663 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:08.795669 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:08.796920 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:08.797353 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:08.797354 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:08.797354 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:08.797406 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:08.797629 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:08.797630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:08.797630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:08.797731 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:08.797784 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:08.798065 (+ 281us) server_negotiation.cc:299] Negotiation successful
0812 16:41:08.798130 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 16:41:08.798372 1480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:08.789206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36741 (local address 127.0.253.195:47495)
0812 16:41:08.789399 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:08.789438 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:08.789961 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:08.790248 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:08.790251 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:08.790875 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:41:08.791149 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:08.791156 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:08.792051 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:08.792052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:08.793246 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:08.793250 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:08.793594 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:08.793595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:08.793706 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:08.794150 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:41:08.794158 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:41:08.795565 (+ 1407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:41:08.796995 (+ 1430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:08.797002 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:08.797004 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:08.797285 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:08.797467 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:08.797467 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:08.797468 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:08.797569 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:08.797788 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:41:08.797792 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:41:08.797944 (+ 152us) client_negotiation.cc:769] Sending connection context
0812 16:41:08.798030 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 16:41:08.798152 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":94,"threads_started":1}
I20260812 16:41:08.798844 1477 heartbeater.cc:347] Connected to a master server at 127.0.253.254:36741
I20260812 16:41:08.798961 1477 heartbeater.cc:464] Registering TS with master...
I20260812 16:41:08.799204 1477 heartbeater.cc:511] Master 127.0.253.254:36741 requested a full tablet report, sending...
I20260812 16:41:08.799584 465 ts_manager.cc:194] Registered new tserver with Master: 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6 (127.0.253.195:38873)
I20260812 16:41:08.799960 465 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.195@KRBTEST.COM'} at 127.0.253.195:47495
I20260812 16:41:08.803164 1015 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:41:08.810045 560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:08.804501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43336 (local address 127.0.253.254:36741)
0812 16:41:08.804581 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:08.804582 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:08.804669 (+ 87us) server_negotiation.cc:407] Connection header received
0812 16:41:08.804692 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:08.804693 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:08.804701 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:08.804746 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:08.805267 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:08.805714 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:08.806387 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:08.806554 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:08.806601 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:08.807751 (+ 1150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:08.807752 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:08.807753 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:08.807758 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:08.809136 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:08.809425 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:08.809426 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:08.809426 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:08.809473 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:08.809619 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:08.809620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:08.809620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:08.809752 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:08.809824 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:08.809907 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 16:41:08.809961 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 16:41:08.811561 465 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43336:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:41:08.813122 465 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:41:08.819895 1502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:08.815915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42922 (local address 127.0.253.194:41953)
0812 16:41:08.816214 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:08.816218 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:08.816227 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:41:08.816260 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:08.816262 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:08.816275 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:08.816610 (+ 335us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:41:08.816691 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:08.817361 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:08.818754 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:08.819451 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:08.819538 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:08.819603 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:08.819775 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 16:41:08.819816 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":75,"threads_started":1}
I20260812 16:41:08.819881 1499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:08.815632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:41953 (local address 127.0.0.1:42922)
0812 16:41:08.816033 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:08.816041 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:08.816106 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:08.816428 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:08.816430 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:08.816433 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:41:08.816519 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:08.816520 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:08.817477 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:08.817478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:08.818660 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:08.818663 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:08.819549 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:08.819551 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:08.819631 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:08.819654 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 16:41:08.819743 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 16:41:08.819800 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"mutex_wait_us":4,"thread_start_us":131,"threads_started":1}
I20260812 16:41:08.820135 1497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:08.815617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.195:38873 (local address 127.0.0.1:50848)
0812 16:41:08.815817 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:08.815859 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:08.815925 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:08.816418 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:08.816421 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:08.816426 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:41:08.816628 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:08.816629 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:08.817943 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:08.817944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:08.818938 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:08.818940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:08.819885 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:08.819886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:08.819951 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:08.819955 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:41:08.819988 (+ 33us) client_negotiation.cc:240] Negotiation successful
0812 16:41:08.820030 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":117,"thread_start_us":146,"threads_started":1}
I20260812 16:41:08.820134 1498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:08.815617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:36977 (local address 127.0.0.1:54384)
0812 16:41:08.815818 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:08.815859 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:08.815925 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:08.816418 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:08.816421 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:08.816424 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:41:08.816548 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:08.816550 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:08.817986 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:08.817987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:08.819026 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:08.819029 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:08.819912 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:08.819912 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:08.819966 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:08.819969 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:41:08.819999 (+ 30us) client_negotiation.cc:240] Negotiation successful
0812 16:41:08.820029 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":26,"thread_start_us":52,"threads_started":1}
I20260812 16:41:08.820415 1500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:08.815759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54384 (local address 127.0.253.193:36977)
0812 16:41:08.816249 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:08.816253 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:08.816261 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:41:08.816299 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:08.816301 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:08.816316 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:08.816716 (+ 400us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:41:08.816802 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:08.817850 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:08.819153 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:08.819822 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:08.820179 (+ 357us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:08.820255 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:08.820284 (+ 29us) server_negotiation.cc:299] Negotiation successful
0812 16:41:08.820329 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":90,"threads_started":1}
I20260812 16:41:08.820464 1503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:08.815999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50848 (local address 127.0.253.195:38873)
0812 16:41:08.816194 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:08.816201 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:08.816211 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:41:08.816255 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:08.816281 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:08.816299 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:08.816524 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:41:08.816712 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:08.817817 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:08.819028 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:08.819773 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:08.820255 (+ 482us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:08.820316 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:08.820339 (+ 23us) server_negotiation.cc:299] Negotiation successful
0812 16:41:08.820377 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":83,"threads_started":1}
I20260812 16:41:08.821128 726 tablet_service.cc:1467] Processing CreateTablet for tablet 0eb40ccb700e48898cec5b0be27c529c (DEFAULT_TABLE table=test-table [id=6c119947c14a483585b73cfa927545d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:41:08.821192 1323 tablet_service.cc:1467] Processing CreateTablet for tablet 0eb40ccb700e48898cec5b0be27c529c (DEFAULT_TABLE table=test-table [id=6c119947c14a483585b73cfa927545d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:41:08.821497 726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0eb40ccb700e48898cec5b0be27c529c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:08.821516 1323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0eb40ccb700e48898cec5b0be27c529c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:08.824414 1505 tablet_bootstrap.cc:492] T 0eb40ccb700e48898cec5b0be27c529c P 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6: Bootstrap starting.
I20260812 16:41:08.824535 1504 tablet_bootstrap.cc:492] T 0eb40ccb700e48898cec5b0be27c529c P 0ed88041ad3048ad8089fe493d305306: Bootstrap starting.
I20260812 16:41:08.825168 1505 tablet_bootstrap.cc:654] T 0eb40ccb700e48898cec5b0be27c529c P 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:08.825177 1504 tablet_bootstrap.cc:654] T 0eb40ccb700e48898cec5b0be27c529c P 0ed88041ad3048ad8089fe493d305306: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:08.825265 1032 tablet_service.cc:1467] Processing CreateTablet for tablet 0eb40ccb700e48898cec5b0be27c529c (DEFAULT_TABLE table=test-table [id=6c119947c14a483585b73cfa927545d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:41:08.825667 1032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0eb40ccb700e48898cec5b0be27c529c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:08.825799 1504 log.cc:824] T 0eb40ccb700e48898cec5b0be27c529c P 0ed88041ad3048ad8089fe493d305306: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:08.825809 1505 log.cc:824] T 0eb40ccb700e48898cec5b0be27c529c P 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:08.826619 1504 tablet_bootstrap.cc:492] T 0eb40ccb700e48898cec5b0be27c529c P 0ed88041ad3048ad8089fe493d305306: No bootstrap required, opened a new log
I20260812 16:41:08.826712 1504 ts_tablet_manager.cc:1397] T 0eb40ccb700e48898cec5b0be27c529c P 0ed88041ad3048ad8089fe493d305306: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:41:08.826891 1505 tablet_bootstrap.cc:492] T 0eb40ccb700e48898cec5b0be27c529c P 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6: No bootstrap required, opened a new log
I20260812 16:41:08.826970 1505 ts_tablet_manager.cc:1397] T 0eb40ccb700e48898cec5b0be27c529c P 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 16:41:08.828439 1504 raft_consensus.cc:348] T 0eb40ccb700e48898cec5b0be27c529c P 0ed88041ad3048ad8089fe493d305306 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e96ebcc70da4cc3ab9f1ed3e6bc6bc6" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38873 } } peers { permanent_uuid: "0ed88041ad3048ad8089fe493d305306" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 36977 } } peers { permanent_uuid: "6283554482d8443eaf5f2ed2288c7701" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 41953 } }
I20260812 16:41:08.828612 1504 raft_consensus.cc:374] T 0eb40ccb700e48898cec5b0be27c529c P 0ed88041ad3048ad8089fe493d305306 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:08.828667 1504 raft_consensus.cc:729] T 0eb40ccb700e48898cec5b0be27c529c P 0ed88041ad3048ad8089fe493d305306 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ed88041ad3048ad8089fe493d305306, State: Initialized, Role: FOLLOWER
I20260812 16:41:08.828787 1504 consensus_queue.cc:260] T 0eb40ccb700e48898cec5b0be27c529c P 0ed88041ad3048ad8089fe493d305306 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e96ebcc70da4cc3ab9f1ed3e6bc6bc6" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38873 } } peers { permanent_uuid: "0ed88041ad3048ad8089fe493d305306" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 36977 } } peers { permanent_uuid: "6283554482d8443eaf5f2ed2288c7701" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 41953 } }
I20260812 16:41:08.829133 1504 ts_tablet_manager.cc:1428] T 0eb40ccb700e48898cec5b0be27c529c P 0ed88041ad3048ad8089fe493d305306: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 16:41:08.829157 874 heartbeater.cc:503] Master 127.0.253.254:36741 was elected leader, sending a full tablet report...
I20260812 16:41:08.829118 1505 raft_consensus.cc:348] T 0eb40ccb700e48898cec5b0be27c529c P 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e96ebcc70da4cc3ab9f1ed3e6bc6bc6" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38873 } } peers { permanent_uuid: "0ed88041ad3048ad8089fe493d305306" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 36977 } } peers { permanent_uuid: "6283554482d8443eaf5f2ed2288c7701" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 41953 } }
I20260812 16:41:08.829286 1505 raft_consensus.cc:374] T 0eb40ccb700e48898cec5b0be27c529c P 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:08.829329 1505 raft_consensus.cc:729] T 0eb40ccb700e48898cec5b0be27c529c P 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6, State: Initialized, Role: FOLLOWER
I20260812 16:41:08.829335 1509 tablet_bootstrap.cc:492] T 0eb40ccb700e48898cec5b0be27c529c P 6283554482d8443eaf5f2ed2288c7701: Bootstrap starting.
I20260812 16:41:08.829447 1505 consensus_queue.cc:260] T 0eb40ccb700e48898cec5b0be27c529c P 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e96ebcc70da4cc3ab9f1ed3e6bc6bc6" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38873 } } peers { permanent_uuid: "0ed88041ad3048ad8089fe493d305306" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 36977 } } peers { permanent_uuid: "6283554482d8443eaf5f2ed2288c7701" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 41953 } }
I20260812 16:41:08.829759 1505 ts_tablet_manager.cc:1428] T 0eb40ccb700e48898cec5b0be27c529c P 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:41:08.829766 1477 heartbeater.cc:503] Master 127.0.253.254:36741 was elected leader, sending a full tablet report...
I20260812 16:41:08.830073 1509 tablet_bootstrap.cc:654] T 0eb40ccb700e48898cec5b0be27c529c P 6283554482d8443eaf5f2ed2288c7701: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:08.830763 1509 log.cc:824] T 0eb40ccb700e48898cec5b0be27c529c P 6283554482d8443eaf5f2ed2288c7701: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:08.831579 1509 tablet_bootstrap.cc:492] T 0eb40ccb700e48898cec5b0be27c529c P 6283554482d8443eaf5f2ed2288c7701: No bootstrap required, opened a new log
I20260812 16:41:08.831661 1509 ts_tablet_manager.cc:1397] T 0eb40ccb700e48898cec5b0be27c529c P 6283554482d8443eaf5f2ed2288c7701: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:41:08.833036 1509 raft_consensus.cc:348] T 0eb40ccb700e48898cec5b0be27c529c P 6283554482d8443eaf5f2ed2288c7701 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e96ebcc70da4cc3ab9f1ed3e6bc6bc6" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38873 } } peers { permanent_uuid: "0ed88041ad3048ad8089fe493d305306" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 36977 } } peers { permanent_uuid: "6283554482d8443eaf5f2ed2288c7701" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 41953 } }
I20260812 16:41:08.833201 1509 raft_consensus.cc:374] T 0eb40ccb700e48898cec5b0be27c529c P 6283554482d8443eaf5f2ed2288c7701 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:08.833250 1509 raft_consensus.cc:729] T 0eb40ccb700e48898cec5b0be27c529c P 6283554482d8443eaf5f2ed2288c7701 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6283554482d8443eaf5f2ed2288c7701, State: Initialized, Role: FOLLOWER
I20260812 16:41:08.833366 1509 consensus_queue.cc:260] T 0eb40ccb700e48898cec5b0be27c529c P 6283554482d8443eaf5f2ed2288c7701 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e96ebcc70da4cc3ab9f1ed3e6bc6bc6" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38873 } } peers { permanent_uuid: "0ed88041ad3048ad8089fe493d305306" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 36977 } } peers { permanent_uuid: "6283554482d8443eaf5f2ed2288c7701" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 41953 } }
I20260812 16:41:08.833667 1509 ts_tablet_manager.cc:1428] T 0eb40ccb700e48898cec5b0be27c529c P 6283554482d8443eaf5f2ed2288c7701: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:41:08.833698 1190 heartbeater.cc:503] Master 127.0.253.254:36741 was elected leader, sending a full tablet report...
W20260812 16:41:08.868700 1192 tablet.cc:2369] T 0eb40ccb700e48898cec5b0be27c529c P 6283554482d8443eaf5f2ed2288c7701: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:41:08.958192 1513 raft_consensus.cc:482] T 0eb40ccb700e48898cec5b0be27c529c P 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:41:08.958379 1513 raft_consensus.cc:504] T 0eb40ccb700e48898cec5b0be27c529c P 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e96ebcc70da4cc3ab9f1ed3e6bc6bc6" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38873 } } peers { permanent_uuid: "0ed88041ad3048ad8089fe493d305306" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 36977 } } peers { permanent_uuid: "6283554482d8443eaf5f2ed2288c7701" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 41953 } }
I20260812 16:41:08.958824 1513 leader_election.cc:288] T 0eb40ccb700e48898cec5b0be27c529c P 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ed88041ad3048ad8089fe493d305306 (127.0.253.193:36977), 6283554482d8443eaf5f2ed2288c7701 (127.0.253.194:41953)
W20260812 16:41:08.959820 876 tablet.cc:2369] T 0eb40ccb700e48898cec5b0be27c529c P 0ed88041ad3048ad8089fe493d305306: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:41:08.962146 1500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:08.959128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:45965 (local address 127.0.253.193:36977)
0812 16:41:08.959185 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:08.959188 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:08.959201 (+ 13us) server_negotiation.cc:407] Connection header received
0812 16:41:08.959304 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:08.959305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:08.959321 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:08.959389 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:41:08.959595 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:08.960340 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:08.961390 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:08.961769 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:08.961857 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:08.961866 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:08.962037 (+ 171us) server_negotiation.cc:299] Negotiation successful
0812 16:41:08.962081 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 16:41:08.962146 1480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:08.959024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:36977 (local address 127.0.253.195:45965)
0812 16:41:08.959129 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:08.959142 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:08.959200 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:08.959411 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:08.959412 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:08.959414 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:41:08.959515 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:08.959517 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:08.960444 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:08.960445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:08.961283 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:08.961285 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:08.961857 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:08.961858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:08.961942 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:08.961951 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 16:41:08.962026 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 16:41:08.962071 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 16:41:08.962257 1502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:08.959210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:33415 (local address 127.0.253.194:41953)
0812 16:41:08.959256 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:08.959258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:08.959400 (+ 142us) server_negotiation.cc:407] Connection header received
0812 16:41:08.959435 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:08.959436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:08.959449 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:08.959514 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:41:08.959745 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:08.960445 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:08.961478 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:08.961867 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:08.961969 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:08.961978 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:08.962143 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 16:41:08.962187 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 16:41:08.962284 1519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:08.959116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:41953 (local address 127.0.253.195:33415)
0812 16:41:08.959306 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:08.959317 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:08.959380 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:08.959536 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:08.959537 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:08.959539 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:41:08.959631 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:08.959633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:08.960601 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:08.960602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:08.961377 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:08.961379 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:08.961972 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:08.961973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:08.962049 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:08.962056 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 16:41:08.962123 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 16:41:08.962166 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":122,"threads_started":1}
I20260812 16:41:08.962618 781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eb40ccb700e48898cec5b0be27c529c" candidate_uuid: "5e96ebcc70da4cc3ab9f1ed3e6bc6bc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ed88041ad3048ad8089fe493d305306" is_pre_election: true
I20260812 16:41:08.962658 1088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eb40ccb700e48898cec5b0be27c529c" candidate_uuid: "5e96ebcc70da4cc3ab9f1ed3e6bc6bc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6283554482d8443eaf5f2ed2288c7701" is_pre_election: true
I20260812 16:41:08.962797 1088 raft_consensus.cc:2454] T 0eb40ccb700e48898cec5b0be27c529c P 6283554482d8443eaf5f2ed2288c7701 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6 in term 0.
I20260812 16:41:08.962798 781 raft_consensus.cc:2454] T 0eb40ccb700e48898cec5b0be27c529c P 0ed88041ad3048ad8089fe493d305306 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6 in term 0.
I20260812 16:41:08.963064 1239 leader_election.cc:302] T 0eb40ccb700e48898cec5b0be27c529c P 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6, 6283554482d8443eaf5f2ed2288c7701; no voters:
I20260812 16:41:08.963233 1513 raft_consensus.cc:2781] T 0eb40ccb700e48898cec5b0be27c529c P 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:41:08.963317 1513 raft_consensus.cc:482] T 0eb40ccb700e48898cec5b0be27c529c P 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:41:08.963336 1513 raft_consensus.cc:3037] T 0eb40ccb700e48898cec5b0be27c529c P 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:08.964108 1513 raft_consensus.cc:504] T 0eb40ccb700e48898cec5b0be27c529c P 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e96ebcc70da4cc3ab9f1ed3e6bc6bc6" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38873 } } peers { permanent_uuid: "0ed88041ad3048ad8089fe493d305306" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 36977 } } peers { permanent_uuid: "6283554482d8443eaf5f2ed2288c7701" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 41953 } }
I20260812 16:41:08.964260 1513 leader_election.cc:288] T 0eb40ccb700e48898cec5b0be27c529c P 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6 [CANDIDATE]: Term 1 election: Requested vote from peers 0ed88041ad3048ad8089fe493d305306 (127.0.253.193:36977), 6283554482d8443eaf5f2ed2288c7701 (127.0.253.194:41953)
I20260812 16:41:08.964416 781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eb40ccb700e48898cec5b0be27c529c" candidate_uuid: "5e96ebcc70da4cc3ab9f1ed3e6bc6bc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ed88041ad3048ad8089fe493d305306"
I20260812 16:41:08.964428 1088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eb40ccb700e48898cec5b0be27c529c" candidate_uuid: "5e96ebcc70da4cc3ab9f1ed3e6bc6bc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6283554482d8443eaf5f2ed2288c7701"
I20260812 16:41:08.964512 1088 raft_consensus.cc:3037] T 0eb40ccb700e48898cec5b0be27c529c P 6283554482d8443eaf5f2ed2288c7701 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:08.964514 781 raft_consensus.cc:3037] T 0eb40ccb700e48898cec5b0be27c529c P 0ed88041ad3048ad8089fe493d305306 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:08.965304 1088 raft_consensus.cc:2454] T 0eb40ccb700e48898cec5b0be27c529c P 6283554482d8443eaf5f2ed2288c7701 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6 in term 1.
I20260812 16:41:08.965473 1239 leader_election.cc:302] T 0eb40ccb700e48898cec5b0be27c529c P 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6, 6283554482d8443eaf5f2ed2288c7701; no voters:
I20260812 16:41:08.965561 1513 raft_consensus.cc:2781] T 0eb40ccb700e48898cec5b0be27c529c P 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:41:08.965652 781 raft_consensus.cc:2454] T 0eb40ccb700e48898cec5b0be27c529c P 0ed88041ad3048ad8089fe493d305306 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6 in term 1.
I20260812 16:41:08.965713 1513 raft_consensus.cc:686] T 0eb40ccb700e48898cec5b0be27c529c P 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6 [term 1 LEADER]: Becoming Leader. State: Replica: 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6, State: Running, Role: LEADER
I20260812 16:41:08.965812 1513 consensus_queue.cc:237] T 0eb40ccb700e48898cec5b0be27c529c P 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e96ebcc70da4cc3ab9f1ed3e6bc6bc6" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38873 } } peers { permanent_uuid: "0ed88041ad3048ad8089fe493d305306" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 36977 } } peers { permanent_uuid: "6283554482d8443eaf5f2ed2288c7701" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 41953 } }
I20260812 16:41:08.969720 465 catalog_manager.cc:5495] T 0eb40ccb700e48898cec5b0be27c529c P 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6 (127.0.253.195). New cstate: current_term: 1 leader_uuid: "5e96ebcc70da4cc3ab9f1ed3e6bc6bc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e96ebcc70da4cc3ab9f1ed3e6bc6bc6" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 38873 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ed88041ad3048ad8089fe493d305306" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 36977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6283554482d8443eaf5f2ed2288c7701" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 41953 } health_report { overall_health: UNKNOWN } } }
I20260812 16:41:09.001298 1503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:08.998732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50852 (local address 127.0.253.195:38873)
0812 16:41:08.998794 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:08.998796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:08.998806 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:41:08.998845 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:08.998846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:08.998864 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:08.998927 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:41:08.999136 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:08.999619 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:09.000430 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:09.000645 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:09.000713 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:09.000860 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:41:09.000965 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:41:09.001029 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:09.001148 (+ 119us) server_negotiation.cc:299] Negotiation successful
0812 16:41:09.001168 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 16:41:09.003394 1088 raft_consensus.cc:1261] T 0eb40ccb700e48898cec5b0be27c529c P 6283554482d8443eaf5f2ed2288c7701 [term 1 FOLLOWER]: Refusing update from remote peer 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:41:09.003413 781 raft_consensus.cc:1261] T 0eb40ccb700e48898cec5b0be27c529c P 0ed88041ad3048ad8089fe493d305306 [term 1 FOLLOWER]: Refusing update from remote peer 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:41:09.003845 1520 consensus_queue.cc:1035] T 0eb40ccb700e48898cec5b0be27c529c P 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6283554482d8443eaf5f2ed2288c7701" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:41:09.003986 1513 consensus_queue.cc:1035] T 0eb40ccb700e48898cec5b0be27c529c P 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ed88041ad3048ad8089fe493d305306" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 36977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:41:09.006343 1532 mvcc.cc:205] Tried to move back new op lower bound from 7317720551434702848 to 7317720551284817920. Current Snapshot: MvccSnapshot[applied={T|T < 7317720551434702848}]
I20260812 16:41:09.007540 1537 mvcc.cc:205] Tried to move back new op lower bound from 7317720551434702848 to 7317720551284817920. Current Snapshot: MvccSnapshot[applied={T|T < 7317720551434702848 or (T in {7317720551434702848})}]
I20260812 16:41:09.008910 465 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:41:09.009016 465 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:41:09.009856 465 catalog_manager.cc:5782] T 00000000000000000000000000000000 P edea5da71d5c45ec8f448260af5138c1: Sending DeleteTablet for 3 replicas of tablet 0eb40ccb700e48898cec5b0be27c529c
I20260812 16:41:09.010179 1323 tablet_service.cc:1514] Processing DeleteTablet for tablet 0eb40ccb700e48898cec5b0be27c529c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:41:09 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:50848
I20260812 16:41:09.010245 726 tablet_service.cc:1514] Processing DeleteTablet for tablet 0eb40ccb700e48898cec5b0be27c529c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:41:09 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:54384
I20260812 16:41:09.010272 1032 tablet_service.cc:1514] Processing DeleteTablet for tablet 0eb40ccb700e48898cec5b0be27c529c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:41:09 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:42922
I20260812 16:41:09.010437 1548 tablet_replica.cc:330] T 0eb40ccb700e48898cec5b0be27c529c P 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6: stopping tablet replica
I20260812 16:41:09.010552 1548 raft_consensus.cc:2229] T 0eb40ccb700e48898cec5b0be27c529c P 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:41:09.010677 1548 raft_consensus.cc:2258] T 0eb40ccb700e48898cec5b0be27c529c P 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:41:09.010931 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 610
I20260812 16:41:09.020358 1553 tablet_replica.cc:330] T 0eb40ccb700e48898cec5b0be27c529c P 6283554482d8443eaf5f2ed2288c7701: stopping tablet replica
I20260812 16:41:09.020604 1553 raft_consensus.cc:2229] T 0eb40ccb700e48898cec5b0be27c529c P 6283554482d8443eaf5f2ed2288c7701 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:41:09.020737 1548 ts_tablet_manager.cc:1905] T 0eb40ccb700e48898cec5b0be27c529c P 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:41:09.020828 1553 raft_consensus.cc:2258] T 0eb40ccb700e48898cec5b0be27c529c P 6283554482d8443eaf5f2ed2288c7701 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:41:09.021435 1553 ts_tablet_manager.cc:1905] T 0eb40ccb700e48898cec5b0be27c529c P 6283554482d8443eaf5f2ed2288c7701: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:41:09.022756 1548 ts_tablet_manager.cc:1918] T 0eb40ccb700e48898cec5b0be27c529c P 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:41:09.022897 1548 log.cc:1196] T 0eb40ccb700e48898cec5b0be27c529c P 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-2/wal/wals/0eb40ccb700e48898cec5b0be27c529c
I20260812 16:41:09.023308 1548 ts_tablet_manager.cc:1939] T 0eb40ccb700e48898cec5b0be27c529c P 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6: Deleting consensus metadata
I20260812 16:41:09.023566 1553 ts_tablet_manager.cc:1918] T 0eb40ccb700e48898cec5b0be27c529c P 6283554482d8443eaf5f2ed2288c7701: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:41:09.023693 1553 log.cc:1196] T 0eb40ccb700e48898cec5b0be27c529c P 6283554482d8443eaf5f2ed2288c7701: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552619305671-1015-0/minicluster-data/ts-1/wal/wals/0eb40ccb700e48898cec5b0be27c529c
I20260812 16:41:09.023952 431 catalog_manager.cc:4841] TS 5e96ebcc70da4cc3ab9f1ed3e6bc6bc6 (127.0.253.195:38873): tablet 0eb40ccb700e48898cec5b0be27c529c (table test-table [id=6c119947c14a483585b73cfa927545d2]) successfully deleted
I20260812 16:41:09.024046 1553 ts_tablet_manager.cc:1939] T 0eb40ccb700e48898cec5b0be27c529c P 6283554482d8443eaf5f2ed2288c7701: Deleting consensus metadata
I20260812 16:41:09.024598 435 catalog_manager.cc:4841] TS 6283554482d8443eaf5f2ed2288c7701 (127.0.253.194:41953): tablet 0eb40ccb700e48898cec5b0be27c529c (table test-table [id=6c119947c14a483585b73cfa927545d2]) successfully deleted
W20260812 16:41:09.026283 433 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.253.193:36977 (error 108)
I20260812 16:41:09.026518 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 880
I20260812 16:41:09.031637 1498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:09.031441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:36977 (local address 127.0.0.1:54396)
0812 16:41:09.031523 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:09.031589 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.193:36977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 16:41:09.031841 433 catalog_manager.cc:4584] TS 0ed88041ad3048ad8089fe493d305306 (127.0.253.193:36977): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0eb40ccb700e48898cec5b0be27c529c: Network error: Client connection negotiation failed: client connection to 127.0.253.193:36977: connect: Connection refused (error 111)
I20260812 16:41:09.042146 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 1201
I20260812 16:41:09.054967 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 400
2026-08-12T16:41:09Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2422 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-12T16:41:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:41:09Z Disabled control of system clock
I20260812 16:41:09.096109 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:45995
--webserver_interface=127.0.253.254
--webserver_port=0
--builtin_ntp_servers=127.0.253.212:45531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:45995
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 16:41:09.179245 1568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:09.179502 1568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:09.179538 1568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:09.180684 1568 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:41:09.180753 1568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:09.180776 1568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:09.180809 1568 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:41:09.180858 1568 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:41:09.182219 1568 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:45531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:45995
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:45995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:09.182498 1568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:09.182799 1568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:09.187800 1584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:09.187836 1587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:09.187914 1568 server_base.cc:1034] running on GCE node
W20260812 16:41:09.187826 1591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:09.188264 1568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:09.188509 1568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:09.189651 1568 hybrid_clock.cc:648] HybridClock initialized: now 1786552869189627 us; error 43 us; skew 500 ppm
I20260812 16:41:09.191004 1568 webserver.cc:460] Webserver started at http://127.0.253.254:41649/ using document root <none> and password file <none>
I20260812 16:41:09.191274 1568 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:09.191349 1568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:09.191514 1568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:09.192596 1568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "56d3be30d51f4a6081ed7a1f897c7df6"
format_stamp: "Formatted at 2026-08-12 16:41:09 on dist-test-slave-w4v5"
server_key: "1aa53c9d126f2856dd5ad1efaf5204c7"
server_key_iv: "8062ee2a2de2644c1b5d0fcfa6a56d70"
server_key_version: "encryptionkey@0"
I20260812 16:41:09.192986 1568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "56d3be30d51f4a6081ed7a1f897c7df6"
format_stamp: "Formatted at 2026-08-12 16:41:09 on dist-test-slave-w4v5"
server_key: "1aa53c9d126f2856dd5ad1efaf5204c7"
server_key_iv: "8062ee2a2de2644c1b5d0fcfa6a56d70"
server_key_version: "encryptionkey@0"
I20260812 16:41:09.194336 1568 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:41:09.195182 1607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:09.195360 1568 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:41:09.195463 1568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "56d3be30d51f4a6081ed7a1f897c7df6"
format_stamp: "Formatted at 2026-08-12 16:41:09 on dist-test-slave-w4v5"
server_key: "1aa53c9d126f2856dd5ad1efaf5204c7"
server_key_iv: "8062ee2a2de2644c1b5d0fcfa6a56d70"
server_key_version: "encryptionkey@0"
I20260812 16:41:09.195549 1568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:09.216161 1568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:09.216507 1568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:09.216672 1568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:09.221107 1568 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:45995
I20260812 16:41:09.221160 1729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:45995 every 8 connection(s)
I20260812 16:41:09.221539 1568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:41:09.222581 1737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:09.224951 1737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56d3be30d51f4a6081ed7a1f897c7df6: Bootstrap starting.
I20260812 16:41:09.225586 1737 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56d3be30d51f4a6081ed7a1f897c7df6: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:09.225896 1737 log.cc:824] T 00000000000000000000000000000000 P 56d3be30d51f4a6081ed7a1f897c7df6: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:09.226575 1737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56d3be30d51f4a6081ed7a1f897c7df6: No bootstrap required, opened a new log
I20260812 16:41:09.228243 1737 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56d3be30d51f4a6081ed7a1f897c7df6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56d3be30d51f4a6081ed7a1f897c7df6" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 45995 } }
I20260812 16:41:09.228380 1737 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56d3be30d51f4a6081ed7a1f897c7df6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:09.228403 1737 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56d3be30d51f4a6081ed7a1f897c7df6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56d3be30d51f4a6081ed7a1f897c7df6, State: Initialized, Role: FOLLOWER
I20260812 16:41:09.228574 1737 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56d3be30d51f4a6081ed7a1f897c7df6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56d3be30d51f4a6081ed7a1f897c7df6" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 45995 } }
I20260812 16:41:09.228638 1737 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56d3be30d51f4a6081ed7a1f897c7df6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:41:09.228659 1737 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56d3be30d51f4a6081ed7a1f897c7df6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:41:09.228722 1737 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56d3be30d51f4a6081ed7a1f897c7df6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:09.229261 1737 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56d3be30d51f4a6081ed7a1f897c7df6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56d3be30d51f4a6081ed7a1f897c7df6" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 45995 } }
I20260812 16:41:09.229374 1737 leader_election.cc:302] T 00000000000000000000000000000000 P 56d3be30d51f4a6081ed7a1f897c7df6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56d3be30d51f4a6081ed7a1f897c7df6; no voters:
I20260812 16:41:09.229560 1737 leader_election.cc:288] T 00000000000000000000000000000000 P 56d3be30d51f4a6081ed7a1f897c7df6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:41:09.229637 1740 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56d3be30d51f4a6081ed7a1f897c7df6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:41:09.229816 1737 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56d3be30d51f4a6081ed7a1f897c7df6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:41:09.229837 1740 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56d3be30d51f4a6081ed7a1f897c7df6 [term 1 LEADER]: Becoming Leader. State: Replica: 56d3be30d51f4a6081ed7a1f897c7df6, State: Running, Role: LEADER
I20260812 16:41:09.229931 1740 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56d3be30d51f4a6081ed7a1f897c7df6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56d3be30d51f4a6081ed7a1f897c7df6" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 45995 } }
I20260812 16:41:09.230690 1743 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56d3be30d51f4a6081ed7a1f897c7df6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56d3be30d51f4a6081ed7a1f897c7df6. Latest consensus state: current_term: 1 leader_uuid: "56d3be30d51f4a6081ed7a1f897c7df6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56d3be30d51f4a6081ed7a1f897c7df6" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 45995 } } }
I20260812 16:41:09.230756 1742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56d3be30d51f4a6081ed7a1f897c7df6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56d3be30d51f4a6081ed7a1f897c7df6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56d3be30d51f4a6081ed7a1f897c7df6" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 45995 } } }
I20260812 16:41:09.230794 1743 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56d3be30d51f4a6081ed7a1f897c7df6 [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:09.230803 1742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56d3be30d51f4a6081ed7a1f897c7df6 [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:09.230955 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 1568
I20260812 16:41:09.231033 1760 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:41:09.231066 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:41:09.231323 1015 external_mini_cluster.cc:1428] Setting key 308f16b73845027cf770fbc585782eed
I20260812 16:41:09.231627 1760 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:41:09.235999 1760 catalog_manager.cc:1347] Generated new cluster ID: e64395987be740d4a88da3cb49f023dd
I20260812 16:41:09.236075 1760 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:41:09.237315 1779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:09.232055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56918 (local address 127.0.253.254:45995)
0812 16:41:09.232596 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:09.232605 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:09.232632 (+ 27us) server_negotiation.cc:407] Connection header received
0812 16:41:09.232722 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:09.232727 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:09.232779 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:09.232901 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:09.233160 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:09.234124 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:09.235254 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:09.235772 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:09.235864 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:09.236603 (+ 739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:09.236607 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:09.236607 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:41:09.236815 (+ 208us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:09.236856 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:41:09.236869 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:09.236930 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:09.237037 (+ 107us) server_negotiation.cc:299] Negotiation successful
0812 16:41:09.237112 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":77,"threads_started":1}
I20260812 16:41:09.247113 1760 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:41:09.247804 1760 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:41:09.251868 1760 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56d3be30d51f4a6081ed7a1f897c7df6: Generated new TSK 0
I20260812 16:41:09.252054 1760 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:41:09.258030 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.193:0
--local_ip_for_outbound_sockets=127.0.253.193
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:45995
--builtin_ntp_servers=127.0.253.212:45531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 16:41:09.348537 1781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:09.348773 1781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:09.348796 1781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:09.350009 1781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:09.350054 1781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:09.350080 1781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.193
I20260812 16:41:09.351521 1781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:45531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:45995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.193
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:09.351768 1781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:09.352097 1781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:09.357362 1791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:09.357391 1795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:09.357504 1781 server_base.cc:1034] running on GCE node
W20260812 16:41:09.357590 1790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:09.357934 1781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:09.358204 1781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:09.359370 1781 hybrid_clock.cc:648] HybridClock initialized: now 1786552869359356 us; error 31 us; skew 500 ppm
I20260812 16:41:09.360764 1781 webserver.cc:460] Webserver started at http://127.0.253.193:41205/ using document root <none> and password file <none>
I20260812 16:41:09.361032 1781 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:09.361100 1781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:09.361253 1781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:09.362327 1781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-0/data/instance:
uuid: "8003cd7960dc492689b7584a55982553"
format_stamp: "Formatted at 2026-08-12 16:41:09 on dist-test-slave-w4v5"
server_key: "6714bfa95b5fe748328f64c8e69eda1e"
server_key_iv: "a2075845878770b1724c9d9984f37653"
server_key_version: "encryptionkey@0"
I20260812 16:41:09.362706 1781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance:
uuid: "8003cd7960dc492689b7584a55982553"
format_stamp: "Formatted at 2026-08-12 16:41:09 on dist-test-slave-w4v5"
server_key: "6714bfa95b5fe748328f64c8e69eda1e"
server_key_iv: "a2075845878770b1724c9d9984f37653"
server_key_version: "encryptionkey@0"
I20260812 16:41:09.364128 1781 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:41:09.364924 1810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:09.365172 1781 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:41:09.365267 1781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-0/wal
uuid: "8003cd7960dc492689b7584a55982553"
format_stamp: "Formatted at 2026-08-12 16:41:09 on dist-test-slave-w4v5"
server_key: "6714bfa95b5fe748328f64c8e69eda1e"
server_key_iv: "a2075845878770b1724c9d9984f37653"
server_key_version: "encryptionkey@0"
I20260812 16:41:09.365345 1781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:09.392627 1781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:09.392933 1781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:09.393039 1781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:09.393246 1781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:41:09.393620 1781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:41:09.393666 1781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:09.393718 1781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:41:09.393744 1781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:09.399669 1781 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.193:45537
I20260812 16:41:09.399737 2037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.193:45537 every 8 connection(s)
I20260812 16:41:09.400096 1781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
I20260812 16:41:09.405244 1779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:09.400923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:50095 (local address 127.0.253.254:45995)
0812 16:41:09.401008 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:09.401011 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:09.401174 (+ 163us) server_negotiation.cc:407] Connection header received
0812 16:41:09.401668 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:09.401669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:09.401683 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:09.401733 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:09.402395 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:09.402886 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:09.403928 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:09.404112 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:09.404178 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:09.404917 (+ 739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:09.404918 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:09.404918 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:41:09.404925 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:09.404938 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:41:09.404942 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:09.404985 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:09.405098 (+ 113us) server_negotiation.cc:299] Negotiation successful
0812 16:41:09.405130 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 16:41:09.405258 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 1781
I20260812 16:41:09.405333 2046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:09.400858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:45995 (local address 127.0.253.193:50095)
0812 16:41:09.401044 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:09.401086 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:09.401581 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:09.401796 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:09.401805 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:09.401813 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:41:09.402325 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:09.402329 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:09.402976 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:09.402976 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:09.403853 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:09.403856 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:09.404188 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:09.404188 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:09.404295 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:09.404787 (+ 492us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:41:09.404788 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:41:09.404817 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:41:09.404839 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:41:09.404996 (+ 157us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:41:09.405001 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:41:09.405003 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 16:41:09.405072 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 16:41:09.405149 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":94,"threads_started":1}
I20260812 16:41:09.405351 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance
I20260812 16:41:09.405531 1015 external_mini_cluster.cc:1428] Setting key 4d3e95837175cd6218a54ee2ccb4f034
I20260812 16:41:09.405900 2039 heartbeater.cc:347] Connected to a master server at 127.0.253.254:45995
I20260812 16:41:09.406002 2039 heartbeater.cc:464] Registering TS with master...
I20260812 16:41:09.406189 2039 heartbeater.cc:511] Master 127.0.253.254:45995 requested a full tablet report, sending...
I20260812 16:41:09.406598 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.194:0
--local_ip_for_outbound_sockets=127.0.253.194
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:45995
--builtin_ntp_servers=127.0.253.212:45531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 16:41:09.406817 1650 ts_manager.cc:194] Registered new tserver with Master: 8003cd7960dc492689b7584a55982553 (127.0.253.193:45537)
I20260812 16:41:09.407473 1650 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.253.193:50095
W20260812 16:41:09.489300 2047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:09.489506 2047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:09.489526 2047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:09.490602 2047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:09.490635 2047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:09.490659 2047 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.194
I20260812 16:41:09.491940 2047 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:45531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:45995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.194
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:09.492128 2047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:09.492372 2047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:09.497545 2061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:09.497550 2062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:09.497561 2066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:09.497792 2047 server_base.cc:1034] running on GCE node
I20260812 16:41:09.498066 2047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:09.498368 2047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:09.499531 2047 hybrid_clock.cc:648] HybridClock initialized: now 1786552869499507 us; error 60 us; skew 500 ppm
I20260812 16:41:09.501012 2047 webserver.cc:460] Webserver started at http://127.0.253.194:39313/ using document root <none> and password file <none>
I20260812 16:41:09.501272 2047 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:09.501348 2047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:09.501502 2047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:09.502736 2047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-1/data/instance:
uuid: "6f1abcc8d5a84e41a7b37e5dfd5ccf15"
format_stamp: "Formatted at 2026-08-12 16:41:09 on dist-test-slave-w4v5"
server_key: "29538869b0b7d426f5a6dd52ce1fc4b9"
server_key_iv: "98bdfc8fd752c8e83aee39cf18a58ab1"
server_key_version: "encryptionkey@0"
I20260812 16:41:09.503180 2047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance:
uuid: "6f1abcc8d5a84e41a7b37e5dfd5ccf15"
format_stamp: "Formatted at 2026-08-12 16:41:09 on dist-test-slave-w4v5"
server_key: "29538869b0b7d426f5a6dd52ce1fc4b9"
server_key_iv: "98bdfc8fd752c8e83aee39cf18a58ab1"
server_key_version: "encryptionkey@0"
I20260812 16:41:09.504557 2047 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:41:09.505378 2075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:09.505559 2047 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:41:09.505702 2047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-1/wal
uuid: "6f1abcc8d5a84e41a7b37e5dfd5ccf15"
format_stamp: "Formatted at 2026-08-12 16:41:09 on dist-test-slave-w4v5"
server_key: "29538869b0b7d426f5a6dd52ce1fc4b9"
server_key_iv: "98bdfc8fd752c8e83aee39cf18a58ab1"
server_key_version: "encryptionkey@0"
I20260812 16:41:09.505784 2047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:09.518682 2047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:09.518985 2047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:09.519186 2047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:09.519423 2047 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:41:09.519757 2047 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:41:09.519814 2047 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:09.519853 2047 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:41:09.519881 2047 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:09.525527 2047 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.194:40611
I20260812 16:41:09.525609 2308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.194:40611 every 8 connection(s)
I20260812 16:41:09.525954 2047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
I20260812 16:41:09.531661 1779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:09.526769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:44359 (local address 127.0.253.254:45995)
0812 16:41:09.526870 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:09.526872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:09.527098 (+ 226us) server_negotiation.cc:407] Connection header received
0812 16:41:09.527643 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:09.527645 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:09.527658 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:09.527709 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:09.528448 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:09.528953 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:09.530034 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:09.530376 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:09.530450 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:09.531370 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:09.531372 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:09.531372 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:41:09.531377 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:09.531393 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:41:09.531398 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:09.531452 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:09.531546 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 16:41:09.531579 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 16:41:09.531910 2314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:09.526691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:45995 (local address 127.0.253.194:44359)
0812 16:41:09.526940 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:09.526981 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:09.527526 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:09.527790 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:09.527794 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:09.527804 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:41:09.528335 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:09.528341 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:09.529058 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:09.529059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:09.529942 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:09.529944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:09.530459 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:09.530460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:09.530614 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:09.531210 (+ 596us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:41:09.531210 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:41:09.531254 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:41:09.531278 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:41:09.531459 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:41:09.531465 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:41:09.531468 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:41:09.531549 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 16:41:09.531682 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":86,"threads_started":1}
I20260812 16:41:09.532366 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 2047
I20260812 16:41:09.532466 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance
I20260812 16:41:09.532480 2309 heartbeater.cc:347] Connected to a master server at 127.0.253.254:45995
I20260812 16:41:09.532575 2309 heartbeater.cc:464] Registering TS with master...
I20260812 16:41:09.532696 1015 external_mini_cluster.cc:1428] Setting key 0379a2439a9dfe0cdf8cf778e435ee93
I20260812 16:41:09.532789 2309 heartbeater.cc:511] Master 127.0.253.254:45995 requested a full tablet report, sending...
I20260812 16:41:09.533196 1647 ts_manager.cc:194] Registered new tserver with Master: 6f1abcc8d5a84e41a7b37e5dfd5ccf15 (127.0.253.194:40611)
I20260812 16:41:09.533600 1647 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.253.194:44359
I20260812 16:41:09.533604 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.195:0
--local_ip_for_outbound_sockets=127.0.253.195
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:45995
--builtin_ntp_servers=127.0.253.212:45531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 16:41:09.617121 2320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:09.617324 2320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:09.617347 2320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:09.618498 2320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:09.618542 2320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:09.618567 2320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.195
I20260812 16:41:09.619973 2320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:45531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:45995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.195
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:09.620227 2320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:09.620540 2320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:09.625538 2331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:09.625547 2335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:09.625844 2320 server_base.cc:1034] running on GCE node
W20260812 16:41:09.625542 2332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:09.626102 2320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:09.626336 2320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:09.627489 2320 hybrid_clock.cc:648] HybridClock initialized: now 1786552869627477 us; error 27 us; skew 500 ppm
I20260812 16:41:09.628726 2320 webserver.cc:460] Webserver started at http://127.0.253.195:44845/ using document root <none> and password file <none>
I20260812 16:41:09.628988 2320 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:09.629057 2320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:09.629212 2320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:09.630314 2320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-2/data/instance:
uuid: "48985377e5f04996a2480252b6f49d1e"
format_stamp: "Formatted at 2026-08-12 16:41:09 on dist-test-slave-w4v5"
server_key: "781ef3db17234f1639fd98155b32329c"
server_key_iv: "282c52fbcb6310935a18cd54249248db"
server_key_version: "encryptionkey@0"
I20260812 16:41:09.630705 2320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance:
uuid: "48985377e5f04996a2480252b6f49d1e"
format_stamp: "Formatted at 2026-08-12 16:41:09 on dist-test-slave-w4v5"
server_key: "781ef3db17234f1639fd98155b32329c"
server_key_iv: "282c52fbcb6310935a18cd54249248db"
server_key_version: "encryptionkey@0"
I20260812 16:41:09.632153 2320 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:41:09.632887 2345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:09.633060 2320 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:41:09.633176 2320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-2/wal
uuid: "48985377e5f04996a2480252b6f49d1e"
format_stamp: "Formatted at 2026-08-12 16:41:09 on dist-test-slave-w4v5"
server_key: "781ef3db17234f1639fd98155b32329c"
server_key_iv: "282c52fbcb6310935a18cd54249248db"
server_key_version: "encryptionkey@0"
I20260812 16:41:09.633251 2320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:09.646903 2320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:09.647248 2320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:09.647408 2320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:09.647632 2320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:41:09.647965 2320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:41:09.648021 2320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:09.648063 2320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:41:09.648092 2320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:09.654078 2320 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.195:40871
I20260812 16:41:09.654126 2576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.195:40871 every 8 connection(s)
I20260812 16:41:09.654497 2320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
I20260812 16:41:09.659405 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 2320
I20260812 16:41:09.659514 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance
I20260812 16:41:09.659667 1779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:09.655377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:48693 (local address 127.0.253.254:45995)
0812 16:41:09.655462 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:09.655465 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:09.655610 (+ 145us) server_negotiation.cc:407] Connection header received
0812 16:41:09.656095 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:09.656096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:09.656106 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:09.656167 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:09.656808 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:09.657299 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:09.658248 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:09.658427 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:09.658476 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:09.659392 (+ 916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:09.659393 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:09.659393 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:41:09.659397 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:09.659409 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:41:09.659413 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:09.659453 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:09.659548 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 16:41:09.659578 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 16:41:09.659734 1015 external_mini_cluster.cc:1428] Setting key 5234d9f13d09653c13d7b23f711818b6
I20260812 16:41:09.659880 2583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:09.655307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:45995 (local address 127.0.253.195:48693)
0812 16:41:09.655494 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:09.655529 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:09.656006 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:09.656222 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:09.656225 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:09.656234 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:41:09.656740 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:09.656744 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:09.657378 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:09.657378 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:09.658172 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:09.658175 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:09.658496 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:09.658497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:09.658602 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:09.659232 (+ 630us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:41:09.659233 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:41:09.659274 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:41:09.659303 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:41:09.659457 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:41:09.659461 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:41:09.659465 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:41:09.659540 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 16:41:09.659650 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":72,"threads_started":1}
I20260812 16:41:09.660445 2579 heartbeater.cc:347] Connected to a master server at 127.0.253.254:45995
I20260812 16:41:09.660543 2579 heartbeater.cc:464] Registering TS with master...
I20260812 16:41:09.660763 2579 heartbeater.cc:511] Master 127.0.253.254:45995 requested a full tablet report, sending...
I20260812 16:41:09.661144 1647 ts_manager.cc:194] Registered new tserver with Master: 48985377e5f04996a2480252b6f49d1e (127.0.253.195:40871)
I20260812 16:41:09.661513 1647 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.253.195:48693
I20260812 16:41:09.671690 1015 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:41:09.675870 1779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:09.673151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56920 (local address 127.0.253.254:45995)
0812 16:41:09.673227 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:09.673229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:09.673305 (+ 76us) server_negotiation.cc:407] Connection header received
0812 16:41:09.673356 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:09.673356 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:09.673366 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:09.673412 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:09.673554 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:09.674022 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:09.674731 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:09.674903 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:09.674952 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:09.675637 (+ 685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:09.675642 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:09.675642 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:41:09.675646 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:09.675658 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:41:09.675661 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:09.675698 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:09.675751 (+ 53us) server_negotiation.cc:299] Negotiation successful
0812 16:41:09.675770 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 16:41:09.677551 1647 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56920:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:41:09.679083 1647 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:41:09.685456 2607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:09.681663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33236 (local address 127.0.253.193:45537)
0812 16:41:09.681831 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:09.681835 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:09.681844 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:41:09.681887 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:09.681889 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:09.681900 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:09.681990 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:41:09.682098 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:09.682955 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:09.684419 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:09.685122 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:09.685211 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:09.685304 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:09.685332 (+ 28us) server_negotiation.cc:299] Negotiation successful
0812 16:41:09.685374 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":96,"threads_started":1}
I20260812 16:41:09.685446 2598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:09.681245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.195:40871 (local address 127.0.0.1:56860)
0812 16:41:09.681351 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:09.681388 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:09.681460 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:09.682002 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:09.682003 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:09.682006 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:41:09.682051 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:09.682053 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:09.683418 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:09.683419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:09.684219 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:09.684222 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:09.685145 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:09.685160 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:09.685226 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:09.685229 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:41:09.685260 (+ 31us) client_negotiation.cc:240] Negotiation successful
0812 16:41:09.685305 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":61,"threads_started":1}
I20260812 16:41:09.685444 2599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:09.681459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:45537 (local address 127.0.0.1:33236)
0812 16:41:09.681661 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:09.681671 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:09.681716 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:09.681990 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:09.681991 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:09.681993 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:41:09.682040 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:09.682042 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:09.683046 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:09.683047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:09.684325 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:09.684327 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:09.685221 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:09.685222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:09.685274 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:09.685277 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:41:09.685319 (+ 42us) client_negotiation.cc:240] Negotiation successful
0812 16:41:09.685359 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":64,"threads_started":1}
I20260812 16:41:09.685714 2603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:09.681247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:40611 (local address 127.0.0.1:48632)
0812 16:41:09.681564 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:09.681576 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:09.681627 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:09.681920 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:09.681924 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:09.681927 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:41:09.681996 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:09.681998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:09.683353 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:09.683354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:09.684402 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:09.684404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:09.685064 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:09.685065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:09.685141 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:09.685155 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 16:41:09.685226 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 16:41:09.685289 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":105,"threads_started":1}
I20260812 16:41:09.685837 2600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:09.681362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56860 (local address 127.0.253.195:40871)
0812 16:41:09.681840 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:09.681844 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:09.681853 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:41:09.681889 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:09.681891 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:09.681905 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:09.682192 (+ 287us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:41:09.682249 (+ 57us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:09.683324 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:09.684311 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:09.685036 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:09.685404 (+ 368us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:09.685497 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:09.685696 (+ 199us) server_negotiation.cc:299] Negotiation successful
0812 16:41:09.685728 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":87,"threads_started":1}
I20260812 16:41:09.686312 2604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:09.681448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48632 (local address 127.0.253.194:40611)
0812 16:41:09.681625 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:09.681630 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:09.681642 (+ 12us) server_negotiation.cc:407] Connection header received
0812 16:41:09.681729 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:09.681732 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:09.681750 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:09.681828 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:41:09.682075 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:09.683264 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:09.684468 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:09.684970 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:09.685065 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:09.685174 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:09.686176 (+ 1002us) server_negotiation.cc:299] Negotiation successful
0812 16:41:09.686229 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":104,"threads_started":1}
I20260812 16:41:09.686331 1916 tablet_service.cc:1467] Processing CreateTablet for tablet 4fda27df38ff42cb9b926d93b415c4a5 (DEFAULT_TABLE table=test-table [id=b4ec836ed398467595b84590879b9d32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:41:09.686769 1916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fda27df38ff42cb9b926d93b415c4a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:09.687067 2181 tablet_service.cc:1467] Processing CreateTablet for tablet 4fda27df38ff42cb9b926d93b415c4a5 (DEFAULT_TABLE table=test-table [id=b4ec836ed398467595b84590879b9d32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:41:09.687460 2181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fda27df38ff42cb9b926d93b415c4a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:09.690414 2608 tablet_bootstrap.cc:492] T 4fda27df38ff42cb9b926d93b415c4a5 P 8003cd7960dc492689b7584a55982553: Bootstrap starting.
I20260812 16:41:09.691045 2609 tablet_bootstrap.cc:492] T 4fda27df38ff42cb9b926d93b415c4a5 P 6f1abcc8d5a84e41a7b37e5dfd5ccf15: Bootstrap starting.
I20260812 16:41:09.691064 2443 tablet_service.cc:1467] Processing CreateTablet for tablet 4fda27df38ff42cb9b926d93b415c4a5 (DEFAULT_TABLE table=test-table [id=b4ec836ed398467595b84590879b9d32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:41:09.691372 2608 tablet_bootstrap.cc:654] T 4fda27df38ff42cb9b926d93b415c4a5 P 8003cd7960dc492689b7584a55982553: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:09.691450 2443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fda27df38ff42cb9b926d93b415c4a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:09.691656 2609 tablet_bootstrap.cc:654] T 4fda27df38ff42cb9b926d93b415c4a5 P 6f1abcc8d5a84e41a7b37e5dfd5ccf15: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:09.691737 2608 log.cc:824] T 4fda27df38ff42cb9b926d93b415c4a5 P 8003cd7960dc492689b7584a55982553: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:09.691871 2609 log.cc:824] T 4fda27df38ff42cb9b926d93b415c4a5 P 6f1abcc8d5a84e41a7b37e5dfd5ccf15: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:09.692596 2609 tablet_bootstrap.cc:492] T 4fda27df38ff42cb9b926d93b415c4a5 P 6f1abcc8d5a84e41a7b37e5dfd5ccf15: No bootstrap required, opened a new log
I20260812 16:41:09.692605 2608 tablet_bootstrap.cc:492] T 4fda27df38ff42cb9b926d93b415c4a5 P 8003cd7960dc492689b7584a55982553: No bootstrap required, opened a new log
I20260812 16:41:09.692660 2609 ts_tablet_manager.cc:1397] T 4fda27df38ff42cb9b926d93b415c4a5 P 6f1abcc8d5a84e41a7b37e5dfd5ccf15: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:41:09.692668 2608 ts_tablet_manager.cc:1397] T 4fda27df38ff42cb9b926d93b415c4a5 P 8003cd7960dc492689b7584a55982553: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 16:41:09.694478 2614 tablet_bootstrap.cc:492] T 4fda27df38ff42cb9b926d93b415c4a5 P 48985377e5f04996a2480252b6f49d1e: Bootstrap starting.
I20260812 16:41:09.695027 2614 tablet_bootstrap.cc:654] T 4fda27df38ff42cb9b926d93b415c4a5 P 48985377e5f04996a2480252b6f49d1e: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:09.695381 2614 log.cc:824] T 4fda27df38ff42cb9b926d93b415c4a5 P 48985377e5f04996a2480252b6f49d1e: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:09.695326 2609 raft_consensus.cc:348] T 4fda27df38ff42cb9b926d93b415c4a5 P 6f1abcc8d5a84e41a7b37e5dfd5ccf15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f1abcc8d5a84e41a7b37e5dfd5ccf15" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 40611 } } peers { permanent_uuid: "48985377e5f04996a2480252b6f49d1e" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 40871 } } peers { permanent_uuid: "8003cd7960dc492689b7584a55982553" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45537 } }
I20260812 16:41:09.695326 2608 raft_consensus.cc:348] T 4fda27df38ff42cb9b926d93b415c4a5 P 8003cd7960dc492689b7584a55982553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f1abcc8d5a84e41a7b37e5dfd5ccf15" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 40611 } } peers { permanent_uuid: "48985377e5f04996a2480252b6f49d1e" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 40871 } } peers { permanent_uuid: "8003cd7960dc492689b7584a55982553" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45537 } }
I20260812 16:41:09.695508 2608 raft_consensus.cc:374] T 4fda27df38ff42cb9b926d93b415c4a5 P 8003cd7960dc492689b7584a55982553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:09.695508 2609 raft_consensus.cc:374] T 4fda27df38ff42cb9b926d93b415c4a5 P 6f1abcc8d5a84e41a7b37e5dfd5ccf15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:09.695554 2608 raft_consensus.cc:729] T 4fda27df38ff42cb9b926d93b415c4a5 P 8003cd7960dc492689b7584a55982553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8003cd7960dc492689b7584a55982553, State: Initialized, Role: FOLLOWER
I20260812 16:41:09.695554 2609 raft_consensus.cc:729] T 4fda27df38ff42cb9b926d93b415c4a5 P 6f1abcc8d5a84e41a7b37e5dfd5ccf15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f1abcc8d5a84e41a7b37e5dfd5ccf15, State: Initialized, Role: FOLLOWER
I20260812 16:41:09.695698 2609 consensus_queue.cc:260] T 4fda27df38ff42cb9b926d93b415c4a5 P 6f1abcc8d5a84e41a7b37e5dfd5ccf15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f1abcc8d5a84e41a7b37e5dfd5ccf15" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 40611 } } peers { permanent_uuid: "48985377e5f04996a2480252b6f49d1e" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 40871 } } peers { permanent_uuid: "8003cd7960dc492689b7584a55982553" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45537 } }
I20260812 16:41:09.695827 2608 consensus_queue.cc:260] T 4fda27df38ff42cb9b926d93b415c4a5 P 8003cd7960dc492689b7584a55982553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f1abcc8d5a84e41a7b37e5dfd5ccf15" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 40611 } } peers { permanent_uuid: "48985377e5f04996a2480252b6f49d1e" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 40871 } } peers { permanent_uuid: "8003cd7960dc492689b7584a55982553" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45537 } }
I20260812 16:41:09.695979 2609 ts_tablet_manager.cc:1428] T 4fda27df38ff42cb9b926d93b415c4a5 P 6f1abcc8d5a84e41a7b37e5dfd5ccf15: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260812 16:41:09.696080 2608 ts_tablet_manager.cc:1428] T 4fda27df38ff42cb9b926d93b415c4a5 P 8003cd7960dc492689b7584a55982553: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260812 16:41:09.696126 2309 heartbeater.cc:503] Master 127.0.253.254:45995 was elected leader, sending a full tablet report...
I20260812 16:41:09.696156 2614 tablet_bootstrap.cc:492] T 4fda27df38ff42cb9b926d93b415c4a5 P 48985377e5f04996a2480252b6f49d1e: No bootstrap required, opened a new log
I20260812 16:41:09.696192 2039 heartbeater.cc:503] Master 127.0.253.254:45995 was elected leader, sending a full tablet report...
I20260812 16:41:09.696213 2614 ts_tablet_manager.cc:1397] T 4fda27df38ff42cb9b926d93b415c4a5 P 48985377e5f04996a2480252b6f49d1e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:41:09.698626 2614 raft_consensus.cc:348] T 4fda27df38ff42cb9b926d93b415c4a5 P 48985377e5f04996a2480252b6f49d1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f1abcc8d5a84e41a7b37e5dfd5ccf15" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 40611 } } peers { permanent_uuid: "48985377e5f04996a2480252b6f49d1e" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 40871 } } peers { permanent_uuid: "8003cd7960dc492689b7584a55982553" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45537 } }
I20260812 16:41:09.698771 2614 raft_consensus.cc:374] T 4fda27df38ff42cb9b926d93b415c4a5 P 48985377e5f04996a2480252b6f49d1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:09.698810 2614 raft_consensus.cc:729] T 4fda27df38ff42cb9b926d93b415c4a5 P 48985377e5f04996a2480252b6f49d1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48985377e5f04996a2480252b6f49d1e, State: Initialized, Role: FOLLOWER
I20260812 16:41:09.698911 2614 consensus_queue.cc:260] T 4fda27df38ff42cb9b926d93b415c4a5 P 48985377e5f04996a2480252b6f49d1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f1abcc8d5a84e41a7b37e5dfd5ccf15" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 40611 } } peers { permanent_uuid: "48985377e5f04996a2480252b6f49d1e" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 40871 } } peers { permanent_uuid: "8003cd7960dc492689b7584a55982553" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45537 } }
I20260812 16:41:09.699194 2614 ts_tablet_manager.cc:1428] T 4fda27df38ff42cb9b926d93b415c4a5 P 48985377e5f04996a2480252b6f49d1e: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260812 16:41:09.699254 2579 heartbeater.cc:503] Master 127.0.253.254:45995 was elected leader, sending a full tablet report...
W20260812 16:41:09.776552 2313 tablet.cc:2369] T 4fda27df38ff42cb9b926d93b415c4a5 P 6f1abcc8d5a84e41a7b37e5dfd5ccf15: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:41:09.879838 2616 raft_consensus.cc:482] T 4fda27df38ff42cb9b926d93b415c4a5 P 6f1abcc8d5a84e41a7b37e5dfd5ccf15 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:41:09.880034 2616 raft_consensus.cc:504] T 4fda27df38ff42cb9b926d93b415c4a5 P 6f1abcc8d5a84e41a7b37e5dfd5ccf15 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f1abcc8d5a84e41a7b37e5dfd5ccf15" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 40611 } } peers { permanent_uuid: "48985377e5f04996a2480252b6f49d1e" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 40871 } } peers { permanent_uuid: "8003cd7960dc492689b7584a55982553" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45537 } }
I20260812 16:41:09.880503 2616 leader_election.cc:288] T 4fda27df38ff42cb9b926d93b415c4a5 P 6f1abcc8d5a84e41a7b37e5dfd5ccf15 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48985377e5f04996a2480252b6f49d1e (127.0.253.195:40871), 8003cd7960dc492689b7584a55982553 (127.0.253.193:45537)
I20260812 16:41:09.884217 2600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:09.880971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:55049 (local address 127.0.253.195:40871)
0812 16:41:09.881023 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:09.881026 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:09.881035 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:41:09.881072 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:09.881073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:09.881081 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:09.881162 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:41:09.881334 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:09.881865 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:09.883485 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:09.883866 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:09.883924 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:09.883931 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:09.884129 (+ 198us) server_negotiation.cc:299] Negotiation successful
0812 16:41:09.884165 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 16:41:09.884238 2314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:09.880716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.195:40871 (local address 127.0.253.194:55049)
0812 16:41:09.880805 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:09.880819 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:09.880869 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:09.881168 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:09.881169 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:09.881171 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:41:09.881252 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:09.881254 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:09.881964 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:09.881965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:09.883392 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:09.883394 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:09.883960 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:09.883961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:09.884049 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:09.884053 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:41:09.884117 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 16:41:09.884161 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260812 16:41:09.884562 2482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fda27df38ff42cb9b926d93b415c4a5" candidate_uuid: "6f1abcc8d5a84e41a7b37e5dfd5ccf15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48985377e5f04996a2480252b6f49d1e" is_pre_election: true
I20260812 16:41:09.884740 2482 raft_consensus.cc:2454] T 4fda27df38ff42cb9b926d93b415c4a5 P 48985377e5f04996a2480252b6f49d1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f1abcc8d5a84e41a7b37e5dfd5ccf15 in term 0.
I20260812 16:41:09.885012 2076 leader_election.cc:302] T 4fda27df38ff42cb9b926d93b415c4a5 P 6f1abcc8d5a84e41a7b37e5dfd5ccf15 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48985377e5f04996a2480252b6f49d1e, 6f1abcc8d5a84e41a7b37e5dfd5ccf15; no voters:
I20260812 16:41:09.885139 2616 raft_consensus.cc:2781] T 4fda27df38ff42cb9b926d93b415c4a5 P 6f1abcc8d5a84e41a7b37e5dfd5ccf15 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:41:09.885206 2616 raft_consensus.cc:482] T 4fda27df38ff42cb9b926d93b415c4a5 P 6f1abcc8d5a84e41a7b37e5dfd5ccf15 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:41:09.885224 2616 raft_consensus.cc:3037] T 4fda27df38ff42cb9b926d93b415c4a5 P 6f1abcc8d5a84e41a7b37e5dfd5ccf15 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:09.885483 2607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:09.880909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:60281 (local address 127.0.253.193:45537)
0812 16:41:09.880974 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:09.880977 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:09.882011 (+ 1034us) server_negotiation.cc:407] Connection header received
0812 16:41:09.882078 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:09.882079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:09.882095 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:09.882150 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:41:09.882362 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:09.883179 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:09.884435 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:09.885053 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:09.885123 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:09.885131 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:09.885371 (+ 240us) server_negotiation.cc:299] Negotiation successful
0812 16:41:09.885411 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 16:41:09.885492 2619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:09.880720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:45537 (local address 127.0.253.194:60281)
0812 16:41:09.881939 (+ 1219us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:09.881950 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:09.882015 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:09.882200 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:09.882201 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:09.882203 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:41:09.882268 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:09.882270 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:09.883297 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:09.883297 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:09.884317 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:09.884321 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:09.885219 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:09.885220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:09.885304 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:09.885307 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:41:09.885362 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 16:41:09.885405 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1202,"mutex_wait_us":66,"thread_start_us":1077,"threads_started":1}
I20260812 16:41:09.885841 1949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fda27df38ff42cb9b926d93b415c4a5" candidate_uuid: "6f1abcc8d5a84e41a7b37e5dfd5ccf15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8003cd7960dc492689b7584a55982553" is_pre_election: true
I20260812 16:41:09.886010 1949 raft_consensus.cc:2454] T 4fda27df38ff42cb9b926d93b415c4a5 P 8003cd7960dc492689b7584a55982553 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f1abcc8d5a84e41a7b37e5dfd5ccf15 in term 0.
I20260812 16:41:09.886227 2616 raft_consensus.cc:504] T 4fda27df38ff42cb9b926d93b415c4a5 P 6f1abcc8d5a84e41a7b37e5dfd5ccf15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f1abcc8d5a84e41a7b37e5dfd5ccf15" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 40611 } } peers { permanent_uuid: "48985377e5f04996a2480252b6f49d1e" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 40871 } } peers { permanent_uuid: "8003cd7960dc492689b7584a55982553" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45537 } }
I20260812 16:41:09.886451 2616 leader_election.cc:288] T 4fda27df38ff42cb9b926d93b415c4a5 P 6f1abcc8d5a84e41a7b37e5dfd5ccf15 [CANDIDATE]: Term 1 election: Requested vote from peers 48985377e5f04996a2480252b6f49d1e (127.0.253.195:40871), 8003cd7960dc492689b7584a55982553 (127.0.253.193:45537)
I20260812 16:41:09.886521 1949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fda27df38ff42cb9b926d93b415c4a5" candidate_uuid: "6f1abcc8d5a84e41a7b37e5dfd5ccf15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8003cd7960dc492689b7584a55982553"
I20260812 16:41:09.886641 2482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fda27df38ff42cb9b926d93b415c4a5" candidate_uuid: "6f1abcc8d5a84e41a7b37e5dfd5ccf15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48985377e5f04996a2480252b6f49d1e"
I20260812 16:41:09.886628 1949 raft_consensus.cc:3037] T 4fda27df38ff42cb9b926d93b415c4a5 P 8003cd7960dc492689b7584a55982553 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:09.886721 2482 raft_consensus.cc:3037] T 4fda27df38ff42cb9b926d93b415c4a5 P 48985377e5f04996a2480252b6f49d1e [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:09.887636 2482 raft_consensus.cc:2454] T 4fda27df38ff42cb9b926d93b415c4a5 P 48985377e5f04996a2480252b6f49d1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f1abcc8d5a84e41a7b37e5dfd5ccf15 in term 1.
I20260812 16:41:09.887638 1949 raft_consensus.cc:2454] T 4fda27df38ff42cb9b926d93b415c4a5 P 8003cd7960dc492689b7584a55982553 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f1abcc8d5a84e41a7b37e5dfd5ccf15 in term 1.
I20260812 16:41:09.887892 2083 leader_election.cc:302] T 4fda27df38ff42cb9b926d93b415c4a5 P 6f1abcc8d5a84e41a7b37e5dfd5ccf15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f1abcc8d5a84e41a7b37e5dfd5ccf15, 8003cd7960dc492689b7584a55982553; no voters:
I20260812 16:41:09.888017 2616 raft_consensus.cc:2781] T 4fda27df38ff42cb9b926d93b415c4a5 P 6f1abcc8d5a84e41a7b37e5dfd5ccf15 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:41:09.888175 2616 raft_consensus.cc:686] T 4fda27df38ff42cb9b926d93b415c4a5 P 6f1abcc8d5a84e41a7b37e5dfd5ccf15 [term 1 LEADER]: Becoming Leader. State: Replica: 6f1abcc8d5a84e41a7b37e5dfd5ccf15, State: Running, Role: LEADER
I20260812 16:41:09.888273 2616 consensus_queue.cc:237] T 4fda27df38ff42cb9b926d93b415c4a5 P 6f1abcc8d5a84e41a7b37e5dfd5ccf15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f1abcc8d5a84e41a7b37e5dfd5ccf15" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 40611 } } peers { permanent_uuid: "48985377e5f04996a2480252b6f49d1e" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 40871 } } peers { permanent_uuid: "8003cd7960dc492689b7584a55982553" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45537 } }
I20260812 16:41:09.889140 1650 catalog_manager.cc:5495] T 4fda27df38ff42cb9b926d93b415c4a5 P 6f1abcc8d5a84e41a7b37e5dfd5ccf15 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f1abcc8d5a84e41a7b37e5dfd5ccf15 (127.0.253.194). New cstate: current_term: 1 leader_uuid: "6f1abcc8d5a84e41a7b37e5dfd5ccf15" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f1abcc8d5a84e41a7b37e5dfd5ccf15" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48985377e5f04996a2480252b6f49d1e" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 40871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8003cd7960dc492689b7584a55982553" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45537 } health_report { overall_health: UNKNOWN } } }
W20260812 16:41:09.900992 2044 tablet.cc:2369] T 4fda27df38ff42cb9b926d93b415c4a5 P 8003cd7960dc492689b7584a55982553: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 16:41:09.905067 2581 tablet.cc:2369] T 4fda27df38ff42cb9b926d93b415c4a5 P 48985377e5f04996a2480252b6f49d1e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:41:09.912066 2604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:09.909461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48634 (local address 127.0.253.194:40611)
0812 16:41:09.909527 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:09.909530 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:09.909541 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:41:09.909639 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:09.909640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:09.909668 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:09.909747 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:41:09.909884 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:09.910372 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:09.911235 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:09.911433 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:09.911508 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:09.911697 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:41:09.911808 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:41:09.911907 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:09.911933 (+ 26us) server_negotiation.cc:299] Negotiation successful
0812 16:41:09.911951 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 16:41:09.913988 1949 raft_consensus.cc:1261] T 4fda27df38ff42cb9b926d93b415c4a5 P 8003cd7960dc492689b7584a55982553 [term 1 FOLLOWER]: Refusing update from remote peer 6f1abcc8d5a84e41a7b37e5dfd5ccf15: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:41:09.913988 2482 raft_consensus.cc:1261] T 4fda27df38ff42cb9b926d93b415c4a5 P 48985377e5f04996a2480252b6f49d1e [term 1 FOLLOWER]: Refusing update from remote peer 6f1abcc8d5a84e41a7b37e5dfd5ccf15: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:41:09.914359 2626 consensus_queue.cc:1035] T 4fda27df38ff42cb9b926d93b415c4a5 P 6f1abcc8d5a84e41a7b37e5dfd5ccf15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8003cd7960dc492689b7584a55982553" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 45537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:41:09.914502 2616 consensus_queue.cc:1035] T 4fda27df38ff42cb9b926d93b415c4a5 P 6f1abcc8d5a84e41a7b37e5dfd5ccf15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48985377e5f04996a2480252b6f49d1e" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:41:09.916875 2632 mvcc.cc:205] Tried to move back new op lower bound from 7317720555164528640 to 7317720555062833152. Current Snapshot: MvccSnapshot[applied={T|T < 7317720555164528640}]
I20260812 16:41:09.917186 2637 mvcc.cc:205] Tried to move back new op lower bound from 7317720555164528640 to 7317720555062833152. Current Snapshot: MvccSnapshot[applied={T|T < 7317720555164528640}]
I20260812 16:41:09.919337 1647 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:41:09.919433 1647 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:41:09.920225 1647 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 56d3be30d51f4a6081ed7a1f897c7df6: Sending DeleteTablet for 3 replicas of tablet 4fda27df38ff42cb9b926d93b415c4a5
I20260812 16:41:09.920593 2181 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fda27df38ff42cb9b926d93b415c4a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:41:09 UTC) from {username='slave'} at 127.0.0.1:48632
I20260812 16:41:09.920614 2443 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fda27df38ff42cb9b926d93b415c4a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:41:09 UTC) from {username='slave'} at 127.0.0.1:56860
I20260812 16:41:09.920818 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 1781
I20260812 16:41:09.920857 1916 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fda27df38ff42cb9b926d93b415c4a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:41:09 UTC) from {username='slave'} at 127.0.0.1:33236
I20260812 16:41:09.921490 2660 tablet_replica.cc:330] T 4fda27df38ff42cb9b926d93b415c4a5 P 6f1abcc8d5a84e41a7b37e5dfd5ccf15: stopping tablet replica
I20260812 16:41:09.921605 2660 raft_consensus.cc:2229] T 4fda27df38ff42cb9b926d93b415c4a5 P 6f1abcc8d5a84e41a7b37e5dfd5ccf15 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:41:09.921716 2660 raft_consensus.cc:2258] T 4fda27df38ff42cb9b926d93b415c4a5 P 6f1abcc8d5a84e41a7b37e5dfd5ccf15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:41:09.921891 2662 tablet_replica.cc:330] T 4fda27df38ff42cb9b926d93b415c4a5 P 48985377e5f04996a2480252b6f49d1e: stopping tablet replica
I20260812 16:41:09.921996 2662 raft_consensus.cc:2229] T 4fda27df38ff42cb9b926d93b415c4a5 P 48985377e5f04996a2480252b6f49d1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:41:09.922119 2662 raft_consensus.cc:2258] T 4fda27df38ff42cb9b926d93b415c4a5 P 48985377e5f04996a2480252b6f49d1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:41:09.928236 2660 ts_tablet_manager.cc:1905] T 4fda27df38ff42cb9b926d93b415c4a5 P 6f1abcc8d5a84e41a7b37e5dfd5ccf15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:41:09.930755 2660 ts_tablet_manager.cc:1918] T 4fda27df38ff42cb9b926d93b415c4a5 P 6f1abcc8d5a84e41a7b37e5dfd5ccf15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:41:09.930850 2660 log.cc:1196] T 4fda27df38ff42cb9b926d93b415c4a5 P 6f1abcc8d5a84e41a7b37e5dfd5ccf15: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-1/wal/wals/4fda27df38ff42cb9b926d93b415c4a5
I20260812 16:41:09.931259 2660 ts_tablet_manager.cc:1939] T 4fda27df38ff42cb9b926d93b415c4a5 P 6f1abcc8d5a84e41a7b37e5dfd5ccf15: Deleting consensus metadata
I20260812 16:41:09.931799 1613 catalog_manager.cc:4841] TS 6f1abcc8d5a84e41a7b37e5dfd5ccf15 (127.0.253.194:40611): tablet 4fda27df38ff42cb9b926d93b415c4a5 (table test-table [id=b4ec836ed398467595b84590879b9d32]) successfully deleted
I20260812 16:41:09.941196 2662 ts_tablet_manager.cc:1905] T 4fda27df38ff42cb9b926d93b415c4a5 P 48985377e5f04996a2480252b6f49d1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:41:09.943488 2662 ts_tablet_manager.cc:1918] T 4fda27df38ff42cb9b926d93b415c4a5 P 48985377e5f04996a2480252b6f49d1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:41:09.943563 2662 log.cc:1196] T 4fda27df38ff42cb9b926d93b415c4a5 P 48985377e5f04996a2480252b6f49d1e: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552619305671-1015-0/minicluster-data/ts-2/wal/wals/4fda27df38ff42cb9b926d93b415c4a5
I20260812 16:41:09.943809 2662 ts_tablet_manager.cc:1939] T 4fda27df38ff42cb9b926d93b415c4a5 P 48985377e5f04996a2480252b6f49d1e: Deleting consensus metadata
I20260812 16:41:09.944347 1609 catalog_manager.cc:4841] TS 48985377e5f04996a2480252b6f49d1e (127.0.253.195:40871): tablet 4fda27df38ff42cb9b926d93b415c4a5 (table test-table [id=b4ec836ed398467595b84590879b9d32]) successfully deleted
W20260812 16:41:09.957654 1613 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.253.193:45537 (error 108)
I20260812 16:41:09.957886 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 2047
I20260812 16:41:09.963264 2603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:09.962972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:45537 (local address 127.0.0.1:33248)
0812 16:41:09.963154 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:09.963213 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.193:45537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155}
W20260812 16:41:09.963413 1613 catalog_manager.cc:4584] TS 8003cd7960dc492689b7584a55982553 (127.0.253.193:45537): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4fda27df38ff42cb9b926d93b415c4a5: Network error: Client connection negotiation failed: client connection to 127.0.253.193:45537: connect: Connection refused (error 111)
I20260812 16:41:09.975704 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 2320
I20260812 16:41:09.984424 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 1568
2026-08-12T16:41:09Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (924 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:41:10 dist-test-slave-w4v5 krb5kdc[2673](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:41:10 dist-test-slave-w4v5 krb5kdc[2673](info): set up 2 sockets
Aug 12 16:41:10 dist-test-slave-w4v5 krb5kdc[2673](info): commencing operation
krb5kdc: starting...
W20260812 16:41:11.335453 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.324s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:41:11 dist-test-slave-w4v5 krb5kdc[2673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:41:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:41:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
I20260812 16:41:11.466748 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:44145
--webserver_interface=127.0.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:45275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:44145
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:41:11.550956 2708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:11.551321 2708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:11.551362 2708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:11.552752 2708 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:41:11.552831 2708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:11.552858 2708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:11.552889 2708 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:41:11.552915 2708 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:41:11.554556 2708 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:45275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:44145
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:44145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:11.554852 2708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:11.555154 2708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:11.560359 2722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:11.560367 2725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:11.560374 2721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:11.560704 2708 server_base.cc:1034] running on GCE node
I20260812 16:41:11.560994 2708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:11.561280 2708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:11.562455 2708 hybrid_clock.cc:648] HybridClock initialized: now 1786552871562419 us; error 57 us; skew 500 ppm
Aug 12 16:41:11 dist-test-slave-w4v5 krb5kdc[2673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552871, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:41:11.565302 2708 init.cc:377] Logged in from keytab as kudu/127.0.253.254@KRBTEST.COM (short username kudu)
I20260812 16:41:11.565828 2708 webserver.cc:460] Webserver started at http://127.0.253.254:42479/ using document root <none> and password file <none>
I20260812 16:41:11.566073 2708 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:11.566147 2708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:11.566286 2708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:11.567493 2708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "cb071948175f447b831d32d899e12c0c"
format_stamp: "Formatted at 2026-08-12 16:41:11 on dist-test-slave-w4v5"
server_key: "e3d77cf07f2092e21c4724a149a9255f"
server_key_iv: "eb3ceeba06b521df536ffa68f42dbb54"
server_key_version: "encryptionkey@0"
I20260812 16:41:11.567878 2708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "cb071948175f447b831d32d899e12c0c"
format_stamp: "Formatted at 2026-08-12 16:41:11 on dist-test-slave-w4v5"
server_key: "e3d77cf07f2092e21c4724a149a9255f"
server_key_iv: "eb3ceeba06b521df536ffa68f42dbb54"
server_key_version: "encryptionkey@0"
I20260812 16:41:11.569314 2708 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 16:41:11.570236 2736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:11.570511 2708 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:41:11.570681 2708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "cb071948175f447b831d32d899e12c0c"
format_stamp: "Formatted at 2026-08-12 16:41:11 on dist-test-slave-w4v5"
server_key: "e3d77cf07f2092e21c4724a149a9255f"
server_key_iv: "eb3ceeba06b521df536ffa68f42dbb54"
server_key_version: "encryptionkey@0"
I20260812 16:41:11.570771 2708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:11.588356 2708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:11.590404 2708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:11.590596 2708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:11.595119 2708 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:44145
I20260812 16:41:11.595173 2842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:44145 every 8 connection(s)
I20260812 16:41:11.595603 2708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:41:11.596576 2847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:11.599117 2847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb071948175f447b831d32d899e12c0c: Bootstrap starting.
I20260812 16:41:11.599742 2847 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb071948175f447b831d32d899e12c0c: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:11.600005 2847 log.cc:824] T 00000000000000000000000000000000 P cb071948175f447b831d32d899e12c0c: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:11.600759 2847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb071948175f447b831d32d899e12c0c: No bootstrap required, opened a new log
I20260812 16:41:11.602712 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 2708
I20260812 16:41:11.602852 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:41:11.602748 2847 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb071948175f447b831d32d899e12c0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb071948175f447b831d32d899e12c0c" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 44145 } }
I20260812 16:41:11.602962 2847 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb071948175f447b831d32d899e12c0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:11.603009 2847 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb071948175f447b831d32d899e12c0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb071948175f447b831d32d899e12c0c, State: Initialized, Role: FOLLOWER
I20260812 16:41:11.603193 1015 external_mini_cluster.cc:1428] Setting key c9fd56da550ab8c8366d0e8b63830f75
I20260812 16:41:11.603225 2847 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb071948175f447b831d32d899e12c0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb071948175f447b831d32d899e12c0c" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 44145 } }
I20260812 16:41:11.603317 2847 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb071948175f447b831d32d899e12c0c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:41:11.603353 2847 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb071948175f447b831d32d899e12c0c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:41:11.603403 2847 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb071948175f447b831d32d899e12c0c [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:11.604288 2847 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb071948175f447b831d32d899e12c0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb071948175f447b831d32d899e12c0c" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 44145 } }
I20260812 16:41:11.604418 2847 leader_election.cc:302] T 00000000000000000000000000000000 P cb071948175f447b831d32d899e12c0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb071948175f447b831d32d899e12c0c; no voters:
I20260812 16:41:11.604588 2847 leader_election.cc:288] T 00000000000000000000000000000000 P cb071948175f447b831d32d899e12c0c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:41:11.604643 2853 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb071948175f447b831d32d899e12c0c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:41:11.604861 2853 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb071948175f447b831d32d899e12c0c [term 1 LEADER]: Becoming Leader. State: Replica: cb071948175f447b831d32d899e12c0c, State: Running, Role: LEADER
I20260812 16:41:11.604897 2847 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb071948175f447b831d32d899e12c0c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:41:11.605062 2853 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb071948175f447b831d32d899e12c0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb071948175f447b831d32d899e12c0c" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 44145 } }
I20260812 16:41:11.605875 2856 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb071948175f447b831d32d899e12c0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb071948175f447b831d32d899e12c0c. Latest consensus state: current_term: 1 leader_uuid: "cb071948175f447b831d32d899e12c0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb071948175f447b831d32d899e12c0c" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 44145 } } }
I20260812 16:41:11.605983 2856 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb071948175f447b831d32d899e12c0c [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:11.606251 2868 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:41:11.607077 2855 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb071948175f447b831d32d899e12c0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb071948175f447b831d32d899e12c0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb071948175f447b831d32d899e12c0c" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 44145 } } }
I20260812 16:41:11.607172 2868 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:41:11.608032 2855 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb071948175f447b831d32d899e12c0c [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:11.609551 2868 catalog_manager.cc:1347] Generated new cluster ID: 9591a9e3eb0f46fdb4066b464f557359
I20260812 16:41:11.609632 2868 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 16:41:11 dist-test-slave-w4v5 krb5kdc[2673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:41:11.618307 2852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:11.603811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58828 (local address 127.0.253.254:44145)
0812 16:41:11.604549 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:11.604558 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:11.604622 (+ 64us) server_negotiation.cc:407] Connection header received
0812 16:41:11.604759 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:11.604769 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:11.604857 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:11.606084 (+ 1227us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:11.606236 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:11.608529 (+ 2293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:11.609518 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:11.610335 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:11.610459 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:11.613544 (+ 3085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:11.613549 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:11.613549 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:11.613563 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:11.616511 (+ 2948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:11.616982 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:11.616985 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:11.616986 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:11.617114 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:11.617358 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:11.617359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:11.617359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:11.617655 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:11.617806 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:11.617951 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 16:41:11.618089 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":518,"threads_started":1}
I20260812 16:41:11.629657 2868 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:41:11.630249 2868 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:41:11.639252 2868 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb071948175f447b831d32d899e12c0c: Generated new TSK 0
I20260812 16:41:11.639489 2868 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab.
Entry for principal kudu/127.0.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab.
I20260812 16:41:11.670274 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.193:0
--local_ip_for_outbound_sockets=127.0.253.193
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:44145
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:45275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:41:11.757632 2897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:11.757881 2897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:11.757921 2897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:11.759174 2897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:11.759243 2897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:11.759291 2897 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.193
I20260812 16:41:11.760783 2897 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:45275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:44145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.193
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:11.761075 2897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:11.761323 2897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:11.766887 2913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:11.766886 2910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:11.766913 2916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:11.767299 2897 server_base.cc:1034] running on GCE node
I20260812 16:41:11.767473 2897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:11.767675 2897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:11.768819 2897 hybrid_clock.cc:648] HybridClock initialized: now 1786552871768801 us; error 33 us; skew 500 ppm
Aug 12 16:41:11 dist-test-slave-w4v5 krb5kdc[2673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552871, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:41:11.775998 2897 init.cc:377] Logged in from keytab as kudu/127.0.253.193@KRBTEST.COM (short username kudu)
I20260812 16:41:11.776517 2897 webserver.cc:460] Webserver started at http://127.0.253.193:33743/ using document root <none> and password file <none>
I20260812 16:41:11.776763 2897 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:11.776834 2897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:11.776998 2897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:11.778092 2897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-0/data/instance:
uuid: "7c880b1c15fd457e8fbfe66c2fb36694"
format_stamp: "Formatted at 2026-08-12 16:41:11 on dist-test-slave-w4v5"
server_key: "27d878affdb8b6e7f69a89157d680ccb"
server_key_iv: "7a4091e7f8d349e03a991f8834af328d"
server_key_version: "encryptionkey@0"
I20260812 16:41:11.778476 2897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance:
uuid: "7c880b1c15fd457e8fbfe66c2fb36694"
format_stamp: "Formatted at 2026-08-12 16:41:11 on dist-test-slave-w4v5"
server_key: "27d878affdb8b6e7f69a89157d680ccb"
server_key_iv: "7a4091e7f8d349e03a991f8834af328d"
server_key_version: "encryptionkey@0"
I20260812 16:41:11.780030 2897 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:41:11.781093 2934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:11.781340 2897 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:41:11.781464 2897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-0/wal
uuid: "7c880b1c15fd457e8fbfe66c2fb36694"
format_stamp: "Formatted at 2026-08-12 16:41:11 on dist-test-slave-w4v5"
server_key: "27d878affdb8b6e7f69a89157d680ccb"
server_key_iv: "7a4091e7f8d349e03a991f8834af328d"
server_key_version: "encryptionkey@0"
I20260812 16:41:11.781546 2897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:11.795346 2897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:11.797752 2897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:11.797961 2897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:11.798259 2897 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:41:11.798684 2897 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:41:11.798746 2897 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:11.798784 2897 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:41:11.798832 2897 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:11.804927 2897 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.193:33363
I20260812 16:41:11.805042 3171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.193:33363 every 8 connection(s)
I20260812 16:41:11.805343 2897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
I20260812 16:41:11.806479 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 2897
I20260812 16:41:11.806597 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance
I20260812 16:41:11.806859 1015 external_mini_cluster.cc:1428] Setting key 0df25285d7929ccddcb0a33f574226e1
Aug 12 16:41:11 dist-test-slave-w4v5 krb5kdc[2673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552871, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.253.194@KRBTEST.COM; defaulting to no policy
I20260812 16:41:11.816648 2852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:11.806272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:51139 (local address 127.0.253.254:44145)
0812 16:41:11.806373 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:11.806375 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:11.807280 (+ 905us) server_negotiation.cc:407] Connection header received
0812 16:41:11.807908 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:11.807909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:11.807926 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:11.807997 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:11.808967 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:11.809742 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:11.810808 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:11.811028 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:11.811149 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:11.813596 (+ 2447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:11.813598 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:11.813598 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:11.813606 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:11.815188 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:11.815630 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:11.815632 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:11.815632 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:11.815698 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:11.815964 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:11.815965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:11.815966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:11.816088 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:11.816166 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:11.816462 (+ 296us) server_negotiation.cc:299] Negotiation successful
0812 16:41:11.816551 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 16:41:11.816859 3174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:11.806205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:44145 (local address 127.0.253.193:51139)
0812 16:41:11.807143 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:11.807184 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:11.807805 (+ 621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:11.808060 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:11.808063 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:11.808669 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:41:11.808868 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:11.808873 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:11.809850 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:11.809852 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:11.810697 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:11.810699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:11.811152 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:11.811153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:11.811304 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:11.811866 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:41:11.811875 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:41:11.813492 (+ 1617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:41:11.815283 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:11.815291 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:11.815293 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:11.815527 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:11.815769 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:11.815770 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:11.815770 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:11.815886 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:11.816169 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:41:11.816177 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:41:11.816320 (+ 143us) client_negotiation.cc:769] Sending connection context
0812 16:41:11.816426 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 16:41:11.816607 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":853,"thread_start_us":112,"threads_started":1}
I20260812 16:41:11.817430 3172 heartbeater.cc:347] Connected to a master server at 127.0.253.254:44145
I20260812 16:41:11.817546 3172 heartbeater.cc:464] Registering TS with master...
I20260812 16:41:11.817790 3172 heartbeater.cc:511] Master 127.0.253.254:44145 requested a full tablet report, sending...
I20260812 16:41:11.818377 2781 ts_manager.cc:194] Registered new tserver with Master: 7c880b1c15fd457e8fbfe66c2fb36694 (127.0.253.193:33363)
I20260812 16:41:11.819134 2781 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.193@KRBTEST.COM'} at 127.0.253.193:51139
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab.
Entry for principal kudu/127.0.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab.
I20260812 16:41:11.834926 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.194:0
--local_ip_for_outbound_sockets=127.0.253.194
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:44145
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:45275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:41:11.920331 3184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:11.920563 3184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:11.920599 3184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:11.921746 3184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:11.921816 3184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:11.921860 3184 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.194
I20260812 16:41:11.923347 3184 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:45275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:44145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3184
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.194
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:11.923624 3184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:11.923878 3184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:11.929024 3196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:11.929013 3193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:11.929013 3194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:11.929473 3184 server_base.cc:1034] running on GCE node
I20260812 16:41:11.929710 3184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:11.929957 3184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:11.931131 3184 hybrid_clock.cc:648] HybridClock initialized: now 1786552871931108 us; error 40 us; skew 500 ppm
Aug 12 16:41:11 dist-test-slave-w4v5 krb5kdc[2673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552871, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:41:11.933701 3184 init.cc:377] Logged in from keytab as kudu/127.0.253.194@KRBTEST.COM (short username kudu)
I20260812 16:41:11.934170 3184 webserver.cc:460] Webserver started at http://127.0.253.194:32781/ using document root <none> and password file <none>
I20260812 16:41:11.934412 3184 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:11.934458 3184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:11.934593 3184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:11.935752 3184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-1/data/instance:
uuid: "da7e89f6b6d84775a79aaa7d389fa8cd"
format_stamp: "Formatted at 2026-08-12 16:41:11 on dist-test-slave-w4v5"
server_key: "1a179fd4100febb66fdcb3370c21a148"
server_key_iv: "dad00cb1f127476530e217bb0bd94a67"
server_key_version: "encryptionkey@0"
I20260812 16:41:11.936117 3184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance:
uuid: "da7e89f6b6d84775a79aaa7d389fa8cd"
format_stamp: "Formatted at 2026-08-12 16:41:11 on dist-test-slave-w4v5"
server_key: "1a179fd4100febb66fdcb3370c21a148"
server_key_iv: "dad00cb1f127476530e217bb0bd94a67"
server_key_version: "encryptionkey@0"
I20260812 16:41:11.937490 3184 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:41:11.938305 3214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:11.938553 3184 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:41:11.938629 3184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-1/wal
uuid: "da7e89f6b6d84775a79aaa7d389fa8cd"
format_stamp: "Formatted at 2026-08-12 16:41:11 on dist-test-slave-w4v5"
server_key: "1a179fd4100febb66fdcb3370c21a148"
server_key_iv: "dad00cb1f127476530e217bb0bd94a67"
server_key_version: "encryptionkey@0"
I20260812 16:41:11.938735 3184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:11.973043 3184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:11.975402 3184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:11.975529 3184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:11.975736 3184 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:41:11.976100 3184 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:41:11.976189 3184 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:11.976261 3184 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:41:11.976289 3184 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:11.982558 3184 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.194:42947
I20260812 16:41:11.982609 3435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.194:42947 every 8 connection(s)
I20260812 16:41:11.982944 3184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
Aug 12 16:41:11 dist-test-slave-w4v5 krb5kdc[2673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552871, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:41:11.991321 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 3184
I20260812 16:41:11.991451 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance
I20260812 16:41:11.991672 1015 external_mini_cluster.cc:1428] Setting key 303db5fe3a25c19c45f6991d260b8b62
I20260812 16:41:11.993243 2852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:11.983770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:56147 (local address 127.0.253.254:44145)
0812 16:41:11.983819 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:11.983821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:11.984104 (+ 283us) server_negotiation.cc:407] Connection header received
0812 16:41:11.984708 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:11.984710 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:11.984723 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:11.984780 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:11.985838 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:11.986352 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:11.987521 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:11.987726 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:11.987840 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:11.990097 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:11.990098 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:11.990099 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:11.990106 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:11.991485 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:11.992125 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:11.992127 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:11.992127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:11.992207 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:11.992575 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:11.992576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:11.992576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:11.992692 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:11.992775 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:11.993054 (+ 279us) server_negotiation.cc:299] Negotiation successful
0812 16:41:11.993142 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 16:41:11.993454 3438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:11.983762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:44145 (local address 127.0.253.194:56147)
0812 16:41:11.983972 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:11.984010 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:11.984593 (+ 583us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:11.984856 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:11.984860 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:11.985464 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:41:11.985730 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:11.985735 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:11.986458 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:11.986459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:11.987413 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:11.987416 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:11.987853 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:11.987854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:11.987986 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:11.988488 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:41:11.988497 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:41:11.989997 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:41:11.991600 (+ 1603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:11.991607 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:11.991609 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:11.991909 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:11.992291 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:11.992291 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:11.992291 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:11.992395 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:11.992765 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:41:11.992772 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:41:11.992917 (+ 145us) client_negotiation.cc:769] Sending connection context
0812 16:41:11.993032 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 16:41:11.993200 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":119,"threads_started":1}
I20260812 16:41:11.993942 3436 heartbeater.cc:347] Connected to a master server at 127.0.253.254:44145
I20260812 16:41:11.994058 3436 heartbeater.cc:464] Registering TS with master...
I20260812 16:41:11.994274 3436 heartbeater.cc:511] Master 127.0.253.254:44145 requested a full tablet report, sending...
I20260812 16:41:11.994678 2781 ts_manager.cc:194] Registered new tserver with Master: da7e89f6b6d84775a79aaa7d389fa8cd (127.0.253.194:42947)
I20260812 16:41:11.995159 2781 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.194@KRBTEST.COM'} at 127.0.253.194:56147
WARNING: no policy specified for kudu/127.0.253.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab.
Entry for principal kudu/127.0.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab.
I20260812 16:41:12.022960 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.195:0
--local_ip_for_outbound_sockets=127.0.253.195
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:44145
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:45275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:41:12.113699 3441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:12.113909 3441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:12.113933 3441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:12.115021 3441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:12.115056 3441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:12.115144 3441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.195
I20260812 16:41:12.116577 3441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:45275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:44145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.195
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:12.116773 3441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:12.116958 3441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:12.122197 3449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:12.122208 3451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:12.122233 3455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:12.122442 3441 server_base.cc:1034] running on GCE node
I20260812 16:41:12.122648 3441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:12.122963 3441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:12.124145 3441 hybrid_clock.cc:648] HybridClock initialized: now 1786552872124130 us; error 44 us; skew 500 ppm
Aug 12 16:41:12 dist-test-slave-w4v5 krb5kdc[2673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552872, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:41:12.126765 3441 init.cc:377] Logged in from keytab as kudu/127.0.253.195@KRBTEST.COM (short username kudu)
I20260812 16:41:12.127279 3441 webserver.cc:460] Webserver started at http://127.0.253.195:38643/ using document root <none> and password file <none>
I20260812 16:41:12.127506 3441 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:12.127575 3441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:12.127732 3441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:12.128808 3441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-2/data/instance:
uuid: "14b0739739cc4180af951a3a53fbcd3b"
format_stamp: "Formatted at 2026-08-12 16:41:12 on dist-test-slave-w4v5"
server_key: "66f234169216fb997cbd1899a5cb9809"
server_key_iv: "8722ee257efaddabd60ebe2bb7cca42f"
server_key_version: "encryptionkey@0"
I20260812 16:41:12.129184 3441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance:
uuid: "14b0739739cc4180af951a3a53fbcd3b"
format_stamp: "Formatted at 2026-08-12 16:41:12 on dist-test-slave-w4v5"
server_key: "66f234169216fb997cbd1899a5cb9809"
server_key_iv: "8722ee257efaddabd60ebe2bb7cca42f"
server_key_version: "encryptionkey@0"
I20260812 16:41:12.130582 3441 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:41:12.131512 3472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:12.131708 3441 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:41:12.131829 3441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-2/wal
uuid: "14b0739739cc4180af951a3a53fbcd3b"
format_stamp: "Formatted at 2026-08-12 16:41:12 on dist-test-slave-w4v5"
server_key: "66f234169216fb997cbd1899a5cb9809"
server_key_iv: "8722ee257efaddabd60ebe2bb7cca42f"
server_key_version: "encryptionkey@0"
I20260812 16:41:12.131906 3441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:12.152411 3441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:12.154436 3441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:12.154621 3441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:12.154889 3441 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:41:12.155287 3441 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:41:12.155344 3441 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:12.155408 3441 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:41:12.155458 3441 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:12.161701 3441 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.195:41975
I20260812 16:41:12.161806 3697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.195:41975 every 8 connection(s)
I20260812 16:41:12.162130 3441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
Aug 12 16:41:12 dist-test-slave-w4v5 krb5kdc[2673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552872, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:41:12.169759 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 3441
I20260812 16:41:12.169888 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance
I20260812 16:41:12.170100 1015 external_mini_cluster.cc:1428] Setting key 4cd81e3cb83cd1b3569732b38fe1b223
I20260812 16:41:12.172722 2852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:12.163006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:53699 (local address 127.0.253.254:44145)
0812 16:41:12.163126 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:12.163129 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:12.163288 (+ 159us) server_negotiation.cc:407] Connection header received
0812 16:41:12.163854 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:12.163855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:12.163867 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:12.163931 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:12.164944 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:12.165464 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:12.166496 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:12.166746 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:12.166804 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:12.169112 (+ 2308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:12.169113 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:12.169113 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:12.169121 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:12.171167 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:12.171625 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:12.171627 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:12.171627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:12.171694 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:12.172023 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:12.172024 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:12.172024 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:12.172161 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:12.172234 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:12.172560 (+ 326us) server_negotiation.cc:299] Negotiation successful
0812 16:41:12.172656 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 16:41:12.172878 3701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:12.162967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:44145 (local address 127.0.253.195:53699)
0812 16:41:12.163155 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:12.163199 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:12.163689 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:12.163989 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:12.163993 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:12.164643 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:41:12.164852 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:12.164857 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:12.165596 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:12.165597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:12.166407 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:12.166409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:12.166815 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:12.166816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:12.166929 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:12.167455 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:41:12.167464 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:41:12.168989 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:41:12.171250 (+ 2261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:12.171258 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:12.171261 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:12.171523 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:12.171802 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:12.171802 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:12.171803 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:12.171920 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:12.172265 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:41:12.172272 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:41:12.172400 (+ 128us) client_negotiation.cc:769] Sending connection context
0812 16:41:12.172512 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 16:41:12.172647 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":121,"threads_started":1}
I20260812 16:41:12.173425 3699 heartbeater.cc:347] Connected to a master server at 127.0.253.254:44145
I20260812 16:41:12.173513 3699 heartbeater.cc:464] Registering TS with master...
I20260812 16:41:12.173660 3699 heartbeater.cc:511] Master 127.0.253.254:44145 requested a full tablet report, sending...
I20260812 16:41:12.174031 2781 ts_manager.cc:194] Registered new tserver with Master: 14b0739739cc4180af951a3a53fbcd3b (127.0.253.195:41975)
I20260812 16:41:12.174418 2781 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.195@KRBTEST.COM'} at 127.0.253.195:53699
I20260812 16:41:12.182132 1015 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:41:12.190155 2852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:12.183823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58836 (local address 127.0.253.254:44145)
0812 16:41:12.183871 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:12.183875 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:12.183945 (+ 70us) server_negotiation.cc:407] Connection header received
0812 16:41:12.184002 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:12.184003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:12.184013 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:12.184117 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:12.184829 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:12.185394 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:12.186116 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:12.186300 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:12.186353 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:12.187769 (+ 1416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:12.187770 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:12.187770 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:12.187777 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:12.189026 (+ 1249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:12.189447 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:12.189448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:12.189448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:12.189507 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:12.189716 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:12.189716 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:12.189717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:12.189853 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:12.189939 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:12.190034 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 16:41:12.190087 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 16:41:12.191928 2781 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58836:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:41:12.193512 2781 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:41:12.200552 3731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:12.196165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.195:41975 (local address 127.0.0.1:53938)
0812 16:41:12.196731 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:12.196740 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:12.196794 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:12.197036 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:12.197037 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:12.197039 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:41:12.197093 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:12.197095 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:12.198005 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:12.198006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:12.199341 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:12.199344 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:12.200220 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:12.200222 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:12.200304 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:12.200320 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 16:41:12.200407 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 16:41:12.200468 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"mutex_wait_us":3,"thread_start_us":107,"threads_started":1}
I20260812 16:41:12.200587 3734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:12.196418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53938 (local address 127.0.253.195:41975)
0812 16:41:12.196698 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:12.196702 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:12.196802 (+ 100us) server_negotiation.cc:407] Connection header received
0812 16:41:12.196869 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:12.196871 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:12.196887 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:12.196964 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:41:12.197187 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:12.197887 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:12.199441 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:12.200126 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:12.200195 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:12.200239 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:12.200448 (+ 209us) server_negotiation.cc:299] Negotiation successful
0812 16:41:12.200496 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":85,"threads_started":1}
I20260812 16:41:12.200989 3728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:12.196165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:33363 (local address 127.0.0.1:36330)
0812 16:41:12.196598 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:12.196622 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:12.196860 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:12.197142 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:12.197142 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:12.197144 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:41:12.197180 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:12.197181 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:12.198532 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:12.198533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:12.199686 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:12.199689 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:12.200724 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:12.200726 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:12.200793 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:12.200797 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:41:12.200873 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 16:41:12.200913 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":105,"threads_started":1}
I20260812 16:41:12.201337 3733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:12.196418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36330 (local address 127.0.253.193:33363)
0812 16:41:12.196618 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:12.196624 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:12.196684 (+ 60us) server_negotiation.cc:407] Connection header received
0812 16:41:12.196963 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:12.196965 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:12.196979 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:12.197031 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:41:12.197263 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:12.198424 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:12.199777 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:12.200348 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:12.200416 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:12.200473 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:12.201201 (+ 728us) server_negotiation.cc:299] Negotiation successful
0812 16:41:12.201249 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":90,"threads_started":1}
I20260812 16:41:12.201388 3571 tablet_service.cc:1467] Processing CreateTablet for tablet 999467895a854e42a56a28d5e386b3c2 (DEFAULT_TABLE table=test-table [id=2b589f6028eb4140b7da3f5825e755d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:41:12.201668 3727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:12.196146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:42947 (local address 127.0.0.1:34162)
0812 16:41:12.196330 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:12.196374 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:12.196453 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:12.196685 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:12.196688 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:12.196691 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:41:12.196771 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:12.196773 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:12.198289 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:12.198290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:12.199349 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:12.199350 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:12.201225 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:12.201227 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:12.201455 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:12.201460 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:41:12.201516 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 16:41:12.201558 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":68,"threads_started":1}
I20260812 16:41:12.201838 3571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 999467895a854e42a56a28d5e386b3c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:12.201998 3732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:12.196262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34162 (local address 127.0.253.194:42947)
0812 16:41:12.196485 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:12.196490 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:12.196502 (+ 12us) server_negotiation.cc:407] Connection header received
0812 16:41:12.196547 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:12.196549 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:12.196565 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:12.196682 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:41:12.196897 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:12.198173 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:12.200273 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:12.201031 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:12.201106 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:12.201427 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:12.201860 (+ 433us) server_negotiation.cc:299] Negotiation successful
0812 16:41:12.201908 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":94,"threads_started":1}
I20260812 16:41:12.202234 3029 tablet_service.cc:1467] Processing CreateTablet for tablet 999467895a854e42a56a28d5e386b3c2 (DEFAULT_TABLE table=test-table [id=2b589f6028eb4140b7da3f5825e755d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:41:12.202611 3029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 999467895a854e42a56a28d5e386b3c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:12.202665 3306 tablet_service.cc:1467] Processing CreateTablet for tablet 999467895a854e42a56a28d5e386b3c2 (DEFAULT_TABLE table=test-table [id=2b589f6028eb4140b7da3f5825e755d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:41:12.202999 3306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 999467895a854e42a56a28d5e386b3c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:12.204715 3736 tablet_bootstrap.cc:492] T 999467895a854e42a56a28d5e386b3c2 P 14b0739739cc4180af951a3a53fbcd3b: Bootstrap starting.
I20260812 16:41:12.205610 3736 tablet_bootstrap.cc:654] T 999467895a854e42a56a28d5e386b3c2 P 14b0739739cc4180af951a3a53fbcd3b: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:12.206270 3736 log.cc:824] T 999467895a854e42a56a28d5e386b3c2 P 14b0739739cc4180af951a3a53fbcd3b: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:12.206642 3738 tablet_bootstrap.cc:492] T 999467895a854e42a56a28d5e386b3c2 P 7c880b1c15fd457e8fbfe66c2fb36694: Bootstrap starting.
I20260812 16:41:12.207058 3736 tablet_bootstrap.cc:492] T 999467895a854e42a56a28d5e386b3c2 P 14b0739739cc4180af951a3a53fbcd3b: No bootstrap required, opened a new log
I20260812 16:41:12.207211 3736 ts_tablet_manager.cc:1397] T 999467895a854e42a56a28d5e386b3c2 P 14b0739739cc4180af951a3a53fbcd3b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 16:41:12.207257 3740 tablet_bootstrap.cc:492] T 999467895a854e42a56a28d5e386b3c2 P da7e89f6b6d84775a79aaa7d389fa8cd: Bootstrap starting.
I20260812 16:41:12.207371 3738 tablet_bootstrap.cc:654] T 999467895a854e42a56a28d5e386b3c2 P 7c880b1c15fd457e8fbfe66c2fb36694: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:12.207983 3738 log.cc:824] T 999467895a854e42a56a28d5e386b3c2 P 7c880b1c15fd457e8fbfe66c2fb36694: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:12.208042 3740 tablet_bootstrap.cc:654] T 999467895a854e42a56a28d5e386b3c2 P da7e89f6b6d84775a79aaa7d389fa8cd: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:12.208665 3740 log.cc:824] T 999467895a854e42a56a28d5e386b3c2 P da7e89f6b6d84775a79aaa7d389fa8cd: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:12.209051 3738 tablet_bootstrap.cc:492] T 999467895a854e42a56a28d5e386b3c2 P 7c880b1c15fd457e8fbfe66c2fb36694: No bootstrap required, opened a new log
I20260812 16:41:12.209111 3738 ts_tablet_manager.cc:1397] T 999467895a854e42a56a28d5e386b3c2 P 7c880b1c15fd457e8fbfe66c2fb36694: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 16:41:12.209046 3736 raft_consensus.cc:348] T 999467895a854e42a56a28d5e386b3c2 P 14b0739739cc4180af951a3a53fbcd3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7e89f6b6d84775a79aaa7d389fa8cd" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42947 } } peers { permanent_uuid: "14b0739739cc4180af951a3a53fbcd3b" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 41975 } } peers { permanent_uuid: "7c880b1c15fd457e8fbfe66c2fb36694" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 33363 } }
I20260812 16:41:12.209251 3736 raft_consensus.cc:374] T 999467895a854e42a56a28d5e386b3c2 P 14b0739739cc4180af951a3a53fbcd3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:12.209299 3736 raft_consensus.cc:729] T 999467895a854e42a56a28d5e386b3c2 P 14b0739739cc4180af951a3a53fbcd3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14b0739739cc4180af951a3a53fbcd3b, State: Initialized, Role: FOLLOWER
I20260812 16:41:12.209414 3736 consensus_queue.cc:260] T 999467895a854e42a56a28d5e386b3c2 P 14b0739739cc4180af951a3a53fbcd3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7e89f6b6d84775a79aaa7d389fa8cd" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42947 } } peers { permanent_uuid: "14b0739739cc4180af951a3a53fbcd3b" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 41975 } } peers { permanent_uuid: "7c880b1c15fd457e8fbfe66c2fb36694" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 33363 } }
I20260812 16:41:12.209575 3740 tablet_bootstrap.cc:492] T 999467895a854e42a56a28d5e386b3c2 P da7e89f6b6d84775a79aaa7d389fa8cd: No bootstrap required, opened a new log
I20260812 16:41:12.209664 3740 ts_tablet_manager.cc:1397] T 999467895a854e42a56a28d5e386b3c2 P da7e89f6b6d84775a79aaa7d389fa8cd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:41:12.209724 3736 ts_tablet_manager.cc:1428] T 999467895a854e42a56a28d5e386b3c2 P 14b0739739cc4180af951a3a53fbcd3b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 16:41:12.209868 3699 heartbeater.cc:503] Master 127.0.253.254:44145 was elected leader, sending a full tablet report...
I20260812 16:41:12.211462 3738 raft_consensus.cc:348] T 999467895a854e42a56a28d5e386b3c2 P 7c880b1c15fd457e8fbfe66c2fb36694 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7e89f6b6d84775a79aaa7d389fa8cd" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42947 } } peers { permanent_uuid: "14b0739739cc4180af951a3a53fbcd3b" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 41975 } } peers { permanent_uuid: "7c880b1c15fd457e8fbfe66c2fb36694" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 33363 } }
I20260812 16:41:12.211678 3738 raft_consensus.cc:374] T 999467895a854e42a56a28d5e386b3c2 P 7c880b1c15fd457e8fbfe66c2fb36694 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:12.211716 3738 raft_consensus.cc:729] T 999467895a854e42a56a28d5e386b3c2 P 7c880b1c15fd457e8fbfe66c2fb36694 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c880b1c15fd457e8fbfe66c2fb36694, State: Initialized, Role: FOLLOWER
I20260812 16:41:12.211849 3738 consensus_queue.cc:260] T 999467895a854e42a56a28d5e386b3c2 P 7c880b1c15fd457e8fbfe66c2fb36694 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7e89f6b6d84775a79aaa7d389fa8cd" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42947 } } peers { permanent_uuid: "14b0739739cc4180af951a3a53fbcd3b" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 41975 } } peers { permanent_uuid: "7c880b1c15fd457e8fbfe66c2fb36694" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 33363 } }
I20260812 16:41:12.211892 3740 raft_consensus.cc:348] T 999467895a854e42a56a28d5e386b3c2 P da7e89f6b6d84775a79aaa7d389fa8cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7e89f6b6d84775a79aaa7d389fa8cd" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42947 } } peers { permanent_uuid: "14b0739739cc4180af951a3a53fbcd3b" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 41975 } } peers { permanent_uuid: "7c880b1c15fd457e8fbfe66c2fb36694" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 33363 } }
I20260812 16:41:12.212090 3740 raft_consensus.cc:374] T 999467895a854e42a56a28d5e386b3c2 P da7e89f6b6d84775a79aaa7d389fa8cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:12.212138 3740 raft_consensus.cc:729] T 999467895a854e42a56a28d5e386b3c2 P da7e89f6b6d84775a79aaa7d389fa8cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da7e89f6b6d84775a79aaa7d389fa8cd, State: Initialized, Role: FOLLOWER
I20260812 16:41:12.212195 3738 ts_tablet_manager.cc:1428] T 999467895a854e42a56a28d5e386b3c2 P 7c880b1c15fd457e8fbfe66c2fb36694: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:41:12.212220 3172 heartbeater.cc:503] Master 127.0.253.254:44145 was elected leader, sending a full tablet report...
I20260812 16:41:12.212267 3740 consensus_queue.cc:260] T 999467895a854e42a56a28d5e386b3c2 P da7e89f6b6d84775a79aaa7d389fa8cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7e89f6b6d84775a79aaa7d389fa8cd" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42947 } } peers { permanent_uuid: "14b0739739cc4180af951a3a53fbcd3b" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 41975 } } peers { permanent_uuid: "7c880b1c15fd457e8fbfe66c2fb36694" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 33363 } }
I20260812 16:41:12.212517 3740 ts_tablet_manager.cc:1428] T 999467895a854e42a56a28d5e386b3c2 P da7e89f6b6d84775a79aaa7d389fa8cd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:41:12.212612 3436 heartbeater.cc:503] Master 127.0.253.254:44145 was elected leader, sending a full tablet report...
W20260812 16:41:12.233520 3437 tablet.cc:2369] T 999467895a854e42a56a28d5e386b3c2 P da7e89f6b6d84775a79aaa7d389fa8cd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:41:12.272483 3752 raft_consensus.cc:482] T 999467895a854e42a56a28d5e386b3c2 P 14b0739739cc4180af951a3a53fbcd3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:41:12.272689 3752 raft_consensus.cc:504] T 999467895a854e42a56a28d5e386b3c2 P 14b0739739cc4180af951a3a53fbcd3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7e89f6b6d84775a79aaa7d389fa8cd" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42947 } } peers { permanent_uuid: "14b0739739cc4180af951a3a53fbcd3b" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 41975 } } peers { permanent_uuid: "7c880b1c15fd457e8fbfe66c2fb36694" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 33363 } }
I20260812 16:41:12.273077 3752 leader_election.cc:288] T 999467895a854e42a56a28d5e386b3c2 P 14b0739739cc4180af951a3a53fbcd3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da7e89f6b6d84775a79aaa7d389fa8cd (127.0.253.194:42947), 7c880b1c15fd457e8fbfe66c2fb36694 (127.0.253.193:33363)
I20260812 16:41:12.276367 3732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:12.273406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:34167 (local address 127.0.253.194:42947)
0812 16:41:12.273462 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:12.273464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:12.273472 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:41:12.273505 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:12.273506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:12.273519 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:12.273597 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:41:12.273849 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:12.274486 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:12.275618 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:12.275987 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:12.276046 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:12.276053 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:12.276245 (+ 192us) server_negotiation.cc:299] Negotiation successful
0812 16:41:12.276284 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 16:41:12.276350 3701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:12.273304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:42947 (local address 127.0.253.195:34167)
0812 16:41:12.273354 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:12.273368 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:12.273414 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:12.273609 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:12.273610 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:12.273612 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:41:12.273750 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:12.273752 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:12.274580 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:12.274581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:12.275495 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:12.275498 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:12.276073 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:12.276074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:12.276163 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:12.276168 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:41:12.276230 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 16:41:12.276279 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":56}
I20260812 16:41:12.276559 3733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:12.273406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:46265 (local address 127.0.253.193:33363)
0812 16:41:12.273454 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:12.273456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:12.273677 (+ 221us) server_negotiation.cc:407] Connection header received
0812 16:41:12.273718 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:12.273719 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:12.273734 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:12.273809 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:41:12.273984 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:12.274503 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:12.275588 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:12.276083 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:12.276223 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:12.276245 (+ 22us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:12.276409 (+ 164us) server_negotiation.cc:299] Negotiation successful
0812 16:41:12.276464 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 16:41:12.276697 3756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:12.273304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:33363 (local address 127.0.253.195:46265)
0812 16:41:12.273584 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:12.273595 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:12.273657 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:12.273832 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:12.273833 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:12.273835 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:41:12.273906 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:12.273907 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:12.274613 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:12.274613 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:12.275485 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:12.275488 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:12.276248 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:12.276249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:12.276319 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:12.276323 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:41:12.276381 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 16:41:12.276564 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":53,"thread_start_us":141,"threads_started":1}
I20260812 16:41:12.276782 3361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "999467895a854e42a56a28d5e386b3c2" candidate_uuid: "14b0739739cc4180af951a3a53fbcd3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da7e89f6b6d84775a79aaa7d389fa8cd" is_pre_election: true
I20260812 16:41:12.276974 3361 raft_consensus.cc:2454] T 999467895a854e42a56a28d5e386b3c2 P da7e89f6b6d84775a79aaa7d389fa8cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14b0739739cc4180af951a3a53fbcd3b in term 0.
I20260812 16:41:12.277009 3069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "999467895a854e42a56a28d5e386b3c2" candidate_uuid: "14b0739739cc4180af951a3a53fbcd3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c880b1c15fd457e8fbfe66c2fb36694" is_pre_election: true
I20260812 16:41:12.277202 3069 raft_consensus.cc:2454] T 999467895a854e42a56a28d5e386b3c2 P 7c880b1c15fd457e8fbfe66c2fb36694 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14b0739739cc4180af951a3a53fbcd3b in term 0.
I20260812 16:41:12.277279 3478 leader_election.cc:302] T 999467895a854e42a56a28d5e386b3c2 P 14b0739739cc4180af951a3a53fbcd3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14b0739739cc4180af951a3a53fbcd3b, da7e89f6b6d84775a79aaa7d389fa8cd; no voters:
I20260812 16:41:12.277411 3752 raft_consensus.cc:2781] T 999467895a854e42a56a28d5e386b3c2 P 14b0739739cc4180af951a3a53fbcd3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:41:12.277473 3752 raft_consensus.cc:482] T 999467895a854e42a56a28d5e386b3c2 P 14b0739739cc4180af951a3a53fbcd3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:41:12.277490 3752 raft_consensus.cc:3037] T 999467895a854e42a56a28d5e386b3c2 P 14b0739739cc4180af951a3a53fbcd3b [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:12.278203 3752 raft_consensus.cc:504] T 999467895a854e42a56a28d5e386b3c2 P 14b0739739cc4180af951a3a53fbcd3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7e89f6b6d84775a79aaa7d389fa8cd" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42947 } } peers { permanent_uuid: "14b0739739cc4180af951a3a53fbcd3b" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 41975 } } peers { permanent_uuid: "7c880b1c15fd457e8fbfe66c2fb36694" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 33363 } }
I20260812 16:41:12.278352 3752 leader_election.cc:288] T 999467895a854e42a56a28d5e386b3c2 P 14b0739739cc4180af951a3a53fbcd3b [CANDIDATE]: Term 1 election: Requested vote from peers da7e89f6b6d84775a79aaa7d389fa8cd (127.0.253.194:42947), 7c880b1c15fd457e8fbfe66c2fb36694 (127.0.253.193:33363)
I20260812 16:41:12.278565 3361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "999467895a854e42a56a28d5e386b3c2" candidate_uuid: "14b0739739cc4180af951a3a53fbcd3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da7e89f6b6d84775a79aaa7d389fa8cd"
I20260812 16:41:12.278605 3069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "999467895a854e42a56a28d5e386b3c2" candidate_uuid: "14b0739739cc4180af951a3a53fbcd3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c880b1c15fd457e8fbfe66c2fb36694"
I20260812 16:41:12.278697 3069 raft_consensus.cc:3037] T 999467895a854e42a56a28d5e386b3c2 P 7c880b1c15fd457e8fbfe66c2fb36694 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:12.278697 3361 raft_consensus.cc:3037] T 999467895a854e42a56a28d5e386b3c2 P da7e89f6b6d84775a79aaa7d389fa8cd [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:12.279772 3361 raft_consensus.cc:2454] T 999467895a854e42a56a28d5e386b3c2 P da7e89f6b6d84775a79aaa7d389fa8cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14b0739739cc4180af951a3a53fbcd3b in term 1.
I20260812 16:41:12.279772 3069 raft_consensus.cc:2454] T 999467895a854e42a56a28d5e386b3c2 P 7c880b1c15fd457e8fbfe66c2fb36694 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14b0739739cc4180af951a3a53fbcd3b in term 1.
I20260812 16:41:12.280023 3481 leader_election.cc:302] T 999467895a854e42a56a28d5e386b3c2 P 14b0739739cc4180af951a3a53fbcd3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14b0739739cc4180af951a3a53fbcd3b, 7c880b1c15fd457e8fbfe66c2fb36694; no voters:
I20260812 16:41:12.280115 3752 raft_consensus.cc:2781] T 999467895a854e42a56a28d5e386b3c2 P 14b0739739cc4180af951a3a53fbcd3b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:41:12.280308 3752 raft_consensus.cc:686] T 999467895a854e42a56a28d5e386b3c2 P 14b0739739cc4180af951a3a53fbcd3b [term 1 LEADER]: Becoming Leader. State: Replica: 14b0739739cc4180af951a3a53fbcd3b, State: Running, Role: LEADER
I20260812 16:41:12.280431 3752 consensus_queue.cc:237] T 999467895a854e42a56a28d5e386b3c2 P 14b0739739cc4180af951a3a53fbcd3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7e89f6b6d84775a79aaa7d389fa8cd" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42947 } } peers { permanent_uuid: "14b0739739cc4180af951a3a53fbcd3b" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 41975 } } peers { permanent_uuid: "7c880b1c15fd457e8fbfe66c2fb36694" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 33363 } }
I20260812 16:41:12.281448 2781 catalog_manager.cc:5495] T 999467895a854e42a56a28d5e386b3c2 P 14b0739739cc4180af951a3a53fbcd3b reported cstate change: term changed from 0 to 1, leader changed from <none> to 14b0739739cc4180af951a3a53fbcd3b (127.0.253.195). New cstate: current_term: 1 leader_uuid: "14b0739739cc4180af951a3a53fbcd3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7e89f6b6d84775a79aaa7d389fa8cd" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14b0739739cc4180af951a3a53fbcd3b" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 41975 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c880b1c15fd457e8fbfe66c2fb36694" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 33363 } health_report { overall_health: UNKNOWN } } }
I20260812 16:41:12.294615 3734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:12.292215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53952 (local address 127.0.253.195:41975)
0812 16:41:12.292279 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:12.292281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:12.292290 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:41:12.292352 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:12.292353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:12.292371 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:12.292435 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:41:12.292576 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:12.293042 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:12.293831 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:12.294020 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:12.294079 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:12.294244 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:41:12.294335 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:41:12.294412 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:12.294460 (+ 48us) server_negotiation.cc:299] Negotiation successful
0812 16:41:12.294470 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 16:41:12.296764 3361 raft_consensus.cc:1261] T 999467895a854e42a56a28d5e386b3c2 P da7e89f6b6d84775a79aaa7d389fa8cd [term 1 FOLLOWER]: Refusing update from remote peer 14b0739739cc4180af951a3a53fbcd3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:41:12.296789 3069 raft_consensus.cc:1261] T 999467895a854e42a56a28d5e386b3c2 P 7c880b1c15fd457e8fbfe66c2fb36694 [term 1 FOLLOWER]: Refusing update from remote peer 14b0739739cc4180af951a3a53fbcd3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:41:12.297250 3752 consensus_queue.cc:1035] T 999467895a854e42a56a28d5e386b3c2 P 14b0739739cc4180af951a3a53fbcd3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c880b1c15fd457e8fbfe66c2fb36694" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 33363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:41:12.297453 3757 consensus_queue.cc:1035] T 999467895a854e42a56a28d5e386b3c2 P 14b0739739cc4180af951a3a53fbcd3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "da7e89f6b6d84775a79aaa7d389fa8cd" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:41:12.300724 3765 mvcc.cc:205] Tried to move back new op lower bound from 7317720564924551168 to 7317720564861599744. Current Snapshot: MvccSnapshot[applied={T|T < 7317720564924551168}]
I20260812 16:41:12.300864 3764 mvcc.cc:205] Tried to move back new op lower bound from 7317720564924551168 to 7317720564861599744. Current Snapshot: MvccSnapshot[applied={T|T < 7317720564924551168}]
I20260812 16:41:12.303275 2780 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:41:12.303385 2780 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:41:12.304289 2780 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cb071948175f447b831d32d899e12c0c: Sending DeleteTablet for 3 replicas of tablet 999467895a854e42a56a28d5e386b3c2
I20260812 16:41:12.304670 3571 tablet_service.cc:1514] Processing DeleteTablet for tablet 999467895a854e42a56a28d5e386b3c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:41:12 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:53938
I20260812 16:41:12.304694 3306 tablet_service.cc:1514] Processing DeleteTablet for tablet 999467895a854e42a56a28d5e386b3c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:41:12 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:34162
I20260812 16:41:12.304982 3029 tablet_service.cc:1514] Processing DeleteTablet for tablet 999467895a854e42a56a28d5e386b3c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:41:12 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:36330
I20260812 16:41:12.305049 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 2897
I20260812 16:41:12.315241 3781 tablet_replica.cc:330] T 999467895a854e42a56a28d5e386b3c2 P da7e89f6b6d84775a79aaa7d389fa8cd: stopping tablet replica
I20260812 16:41:12.315394 3781 raft_consensus.cc:2229] T 999467895a854e42a56a28d5e386b3c2 P da7e89f6b6d84775a79aaa7d389fa8cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:41:12.315511 3781 raft_consensus.cc:2258] T 999467895a854e42a56a28d5e386b3c2 P da7e89f6b6d84775a79aaa7d389fa8cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:41:12.321921 3781 ts_tablet_manager.cc:1905] T 999467895a854e42a56a28d5e386b3c2 P da7e89f6b6d84775a79aaa7d389fa8cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:41:12.324752 3781 ts_tablet_manager.cc:1918] T 999467895a854e42a56a28d5e386b3c2 P da7e89f6b6d84775a79aaa7d389fa8cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:41:12.324963 3781 log.cc:1196] T 999467895a854e42a56a28d5e386b3c2 P da7e89f6b6d84775a79aaa7d389fa8cd: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-1/wal/wals/999467895a854e42a56a28d5e386b3c2
I20260812 16:41:12.325649 3781 ts_tablet_manager.cc:1939] T 999467895a854e42a56a28d5e386b3c2 P da7e89f6b6d84775a79aaa7d389fa8cd: Deleting consensus metadata
I20260812 16:41:12.326402 2742 catalog_manager.cc:4841] TS da7e89f6b6d84775a79aaa7d389fa8cd (127.0.253.194:42947): tablet 999467895a854e42a56a28d5e386b3c2 (table test-table [id=2b589f6028eb4140b7da3f5825e755d9]) successfully deleted
I20260812 16:41:12.327200 3780 tablet_replica.cc:330] T 999467895a854e42a56a28d5e386b3c2 P 14b0739739cc4180af951a3a53fbcd3b: stopping tablet replica
I20260812 16:41:12.327347 3780 raft_consensus.cc:2229] T 999467895a854e42a56a28d5e386b3c2 P 14b0739739cc4180af951a3a53fbcd3b [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:41:12.327502 3780 raft_consensus.cc:2258] T 999467895a854e42a56a28d5e386b3c2 P 14b0739739cc4180af951a3a53fbcd3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:41:12.328774 3780 ts_tablet_manager.cc:1905] T 999467895a854e42a56a28d5e386b3c2 P 14b0739739cc4180af951a3a53fbcd3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:41:12.330528 3780 ts_tablet_manager.cc:1918] T 999467895a854e42a56a28d5e386b3c2 P 14b0739739cc4180af951a3a53fbcd3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:41:12.330616 3780 log.cc:1196] T 999467895a854e42a56a28d5e386b3c2 P 14b0739739cc4180af951a3a53fbcd3b: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552619305671-1015-0/minicluster-data/ts-2/wal/wals/999467895a854e42a56a28d5e386b3c2
I20260812 16:41:12.330933 3780 ts_tablet_manager.cc:1939] T 999467895a854e42a56a28d5e386b3c2 P 14b0739739cc4180af951a3a53fbcd3b: Deleting consensus metadata
I20260812 16:41:12.331544 2739 catalog_manager.cc:4841] TS 14b0739739cc4180af951a3a53fbcd3b (127.0.253.195:41975): tablet 999467895a854e42a56a28d5e386b3c2 (table test-table [id=2b589f6028eb4140b7da3f5825e755d9]) successfully deleted
W20260812 16:41:12.337167 2743 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.253.193:33363 (error 108)
I20260812 16:41:12.337404 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 3184
I20260812 16:41:12.365870 3727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:12.365701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:33363 (local address 127.0.0.1:36344)
0812 16:41:12.365771 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:12.365828 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.193:33363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 16:41:12.366158 2743 catalog_manager.cc:4584] TS 7c880b1c15fd457e8fbfe66c2fb36694 (127.0.253.193:33363): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 999467895a854e42a56a28d5e386b3c2: Network error: Client connection negotiation failed: client connection to 127.0.253.193:33363: connect: Connection refused (error 111)
I20260812 16:41:12.371228 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 3441
I20260812 16:41:12.381340 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 2708
2026-08-12T16:41:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2400 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-12T16:41:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:41:12Z Disabled control of system clock
I20260812 16:41:12.421902 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:33455
--webserver_interface=127.0.253.254
--webserver_port=0
--builtin_ntp_servers=127.0.253.212:40855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:33455
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 16:41:12.508054 3789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:12.508294 3789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:12.508316 3789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:12.509557 3789 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:41:12.509601 3789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:12.509614 3789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:12.509627 3789 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:41:12.509642 3789 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:41:12.511461 3789 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:40855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:33455
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:33455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:12.511813 3789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:12.512158 3789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:12.517099 3810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:12.517107 3807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:12.517359 3789 server_base.cc:1034] running on GCE node
W20260812 16:41:12.517107 3805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:12.517778 3789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:12.518057 3789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:12.519162 3789 hybrid_clock.cc:648] HybridClock initialized: now 1786552872519140 us; error 42 us; skew 500 ppm
I20260812 16:41:12.520439 3789 webserver.cc:460] Webserver started at http://127.0.253.254:44303/ using document root <none> and password file <none>
I20260812 16:41:12.520673 3789 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:12.520718 3789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:12.520816 3789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:12.521903 3789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "861949ed6896419380ae83d42161671d"
format_stamp: "Formatted at 2026-08-12 16:41:12 on dist-test-slave-w4v5"
server_key: "1b42db002a43d3c30085eb2f7b38292b"
server_key_iv: "24f8ef1809b8f460da082de23b52dd56"
server_key_version: "encryptionkey@0"
I20260812 16:41:12.522212 3789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "861949ed6896419380ae83d42161671d"
format_stamp: "Formatted at 2026-08-12 16:41:12 on dist-test-slave-w4v5"
server_key: "1b42db002a43d3c30085eb2f7b38292b"
server_key_iv: "24f8ef1809b8f460da082de23b52dd56"
server_key_version: "encryptionkey@0"
I20260812 16:41:12.523612 3789 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:41:12.524508 3819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:12.524760 3789 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:41:12.524860 3789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "861949ed6896419380ae83d42161671d"
format_stamp: "Formatted at 2026-08-12 16:41:12 on dist-test-slave-w4v5"
server_key: "1b42db002a43d3c30085eb2f7b38292b"
server_key_iv: "24f8ef1809b8f460da082de23b52dd56"
server_key_version: "encryptionkey@0"
I20260812 16:41:12.524914 3789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:12.544481 3789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:12.544801 3789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:12.544911 3789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:12.549114 3789 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:33455
I20260812 16:41:12.549175 3911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:33455 every 8 connection(s)
I20260812 16:41:12.549553 3789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:41:12.550704 3913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:12.553078 3913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 861949ed6896419380ae83d42161671d: Bootstrap starting.
I20260812 16:41:12.553717 3913 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 861949ed6896419380ae83d42161671d: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:12.554028 3913 log.cc:824] T 00000000000000000000000000000000 P 861949ed6896419380ae83d42161671d: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:12.554730 3913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 861949ed6896419380ae83d42161671d: No bootstrap required, opened a new log
I20260812 16:41:12.556394 3913 raft_consensus.cc:348] T 00000000000000000000000000000000 P 861949ed6896419380ae83d42161671d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861949ed6896419380ae83d42161671d" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 33455 } }
I20260812 16:41:12.556562 3913 raft_consensus.cc:374] T 00000000000000000000000000000000 P 861949ed6896419380ae83d42161671d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:12.556613 3913 raft_consensus.cc:729] T 00000000000000000000000000000000 P 861949ed6896419380ae83d42161671d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 861949ed6896419380ae83d42161671d, State: Initialized, Role: FOLLOWER
I20260812 16:41:12.556789 3913 consensus_queue.cc:260] T 00000000000000000000000000000000 P 861949ed6896419380ae83d42161671d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861949ed6896419380ae83d42161671d" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 33455 } }
I20260812 16:41:12.556893 3913 raft_consensus.cc:388] T 00000000000000000000000000000000 P 861949ed6896419380ae83d42161671d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:41:12.556939 3913 raft_consensus.cc:482] T 00000000000000000000000000000000 P 861949ed6896419380ae83d42161671d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:41:12.556993 3913 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 861949ed6896419380ae83d42161671d [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:12.557339 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 3789
I20260812 16:41:12.557438 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:41:12.557672 1015 external_mini_cluster.cc:1428] Setting key 3168f12a0069f9e92aafc10551120301
I20260812 16:41:12.557755 3913 raft_consensus.cc:504] T 00000000000000000000000000000000 P 861949ed6896419380ae83d42161671d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861949ed6896419380ae83d42161671d" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 33455 } }
I20260812 16:41:12.557870 3913 leader_election.cc:302] T 00000000000000000000000000000000 P 861949ed6896419380ae83d42161671d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 861949ed6896419380ae83d42161671d; no voters:
I20260812 16:41:12.558066 3913 leader_election.cc:288] T 00000000000000000000000000000000 P 861949ed6896419380ae83d42161671d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:41:12.558125 3917 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 861949ed6896419380ae83d42161671d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:41:12.558382 3917 raft_consensus.cc:686] T 00000000000000000000000000000000 P 861949ed6896419380ae83d42161671d [term 1 LEADER]: Becoming Leader. State: Replica: 861949ed6896419380ae83d42161671d, State: Running, Role: LEADER
I20260812 16:41:12.558437 3913 sys_catalog.cc:564] T 00000000000000000000000000000000 P 861949ed6896419380ae83d42161671d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:41:12.558547 3917 consensus_queue.cc:237] T 00000000000000000000000000000000 P 861949ed6896419380ae83d42161671d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861949ed6896419380ae83d42161671d" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 33455 } }
I20260812 16:41:12.559146 3919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 861949ed6896419380ae83d42161671d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 861949ed6896419380ae83d42161671d. Latest consensus state: current_term: 1 leader_uuid: "861949ed6896419380ae83d42161671d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861949ed6896419380ae83d42161671d" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 33455 } } }
I20260812 16:41:12.559247 3919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 861949ed6896419380ae83d42161671d [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:12.559233 3921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 861949ed6896419380ae83d42161671d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "861949ed6896419380ae83d42161671d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861949ed6896419380ae83d42161671d" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 33455 } } }
I20260812 16:41:12.559664 3921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 861949ed6896419380ae83d42161671d [sys.catalog]: This master's current role is: LEADER
W20260812 16:41:12.560673 3947 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 861949ed6896419380ae83d42161671d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 16:41:12.560738 3947 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:41:12.560814 3938 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:41:12.560994 3938 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:41:12.562480 3938 catalog_manager.cc:1347] Generated new cluster ID: 8fc95b56ef9440d7ab2dc530453fd40e
I20260812 16:41:12.562588 3938 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:41:12.564069 3922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:12.558335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34564 (local address 127.0.253.254:33455)
0812 16:41:12.558540 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:12.558549 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:12.558581 (+ 32us) server_negotiation.cc:407] Connection header received
0812 16:41:12.558716 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:12.558724 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:12.558795 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:12.558952 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:12.559234 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:12.560647 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:12.561784 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:12.562427 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:12.562547 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:12.563324 (+ 777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:12.563328 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:12.563328 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:41:12.563531 (+ 203us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:12.563570 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:41:12.563584 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:12.563647 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:12.563769 (+ 122us) server_negotiation.cc:299] Negotiation successful
0812 16:41:12.563848 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":71,"threads_started":1}
I20260812 16:41:12.579878 3938 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:41:12.580632 3938 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:41:12.601068 3938 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 861949ed6896419380ae83d42161671d: Generated new TSK 0
I20260812 16:41:12.601330 3938 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:41:12.610180 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.193:0
--local_ip_for_outbound_sockets=127.0.253.193
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:33455
--builtin_ntp_servers=127.0.253.212:40855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 16:41:12.694732 3953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:12.694945 3953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:12.694967 3953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:12.696193 3953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:12.696242 3953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:12.696269 3953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.193
I20260812 16:41:12.697630 3953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:40855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:33455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.193
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:12.697886 3953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:12.698206 3953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:12.703461 3960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:12.703480 3968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:12.703469 3963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:12.703905 3953 server_base.cc:1034] running on GCE node
I20260812 16:41:12.704130 3953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:12.704416 3953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:12.705577 3953 hybrid_clock.cc:648] HybridClock initialized: now 1786552872705561 us; error 26 us; skew 500 ppm
I20260812 16:41:12.706976 3953 webserver.cc:460] Webserver started at http://127.0.253.193:34331/ using document root <none> and password file <none>
I20260812 16:41:12.707316 3953 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:12.707391 3953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:12.707535 3953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:12.708644 3953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-0/data/instance:
uuid: "6fe9845519d54d358d05947fd7c60c66"
format_stamp: "Formatted at 2026-08-12 16:41:12 on dist-test-slave-w4v5"
server_key: "03aed751f7750b12ac9b817cdaf6d644"
server_key_iv: "c0ec46a1c8e31f907757d4fad6955fd5"
server_key_version: "encryptionkey@0"
I20260812 16:41:12.709049 3953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance:
uuid: "6fe9845519d54d358d05947fd7c60c66"
format_stamp: "Formatted at 2026-08-12 16:41:12 on dist-test-slave-w4v5"
server_key: "03aed751f7750b12ac9b817cdaf6d644"
server_key_iv: "c0ec46a1c8e31f907757d4fad6955fd5"
server_key_version: "encryptionkey@0"
I20260812 16:41:12.710465 3953 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:41:12.711323 3976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:12.711503 3953 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:41:12.711597 3953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-0/wal
uuid: "6fe9845519d54d358d05947fd7c60c66"
format_stamp: "Formatted at 2026-08-12 16:41:12 on dist-test-slave-w4v5"
server_key: "03aed751f7750b12ac9b817cdaf6d644"
server_key_iv: "c0ec46a1c8e31f907757d4fad6955fd5"
server_key_version: "encryptionkey@0"
I20260812 16:41:12.711719 3953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:12.723488 3953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:12.723845 3953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:12.724002 3953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:12.724251 3953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:41:12.724606 3953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:41:12.724674 3953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:12.724720 3953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:41:12.724750 3953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:12.730772 3953 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.193:33321
I20260812 16:41:12.730883 4191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.193:33321 every 8 connection(s)
I20260812 16:41:12.731258 3953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
I20260812 16:41:12.736505 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 3953
I20260812 16:41:12.736622 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance
I20260812 16:41:12.736891 1015 external_mini_cluster.cc:1428] Setting key 2984fd7bdd5f213886b1ab56f0dcfc6e
I20260812 16:41:12.737074 3922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:12.732092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:43627 (local address 127.0.253.254:33455)
0812 16:41:12.732169 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:12.732171 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:12.732444 (+ 273us) server_negotiation.cc:407] Connection header received
0812 16:41:12.733022 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:12.733023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:12.733041 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:12.733093 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:12.733812 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:12.734332 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:12.735411 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:12.735640 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:12.735722 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:12.736648 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:12.736650 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:12.736650 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:41:12.736657 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:12.736676 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:41:12.736690 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:12.736770 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:12.736891 (+ 121us) server_negotiation.cc:299] Negotiation successful
0812 16:41:12.736923 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 16:41:12.737237 4195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:12.732104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:33455 (local address 127.0.253.193:43627)
0812 16:41:12.732318 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:12.732354 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:12.732898 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:12.733180 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:12.733183 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:12.733192 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:41:12.733702 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:12.733705 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:12.734437 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:12.734438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:12.735321 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:12.735324 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:12.735729 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:12.735731 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:12.735863 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:12.736424 (+ 561us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:41:12.736424 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:41:12.736468 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:41:12.736498 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:41:12.736789 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:41:12.736798 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:41:12.736801 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:41:12.736884 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 16:41:12.737001 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":98,"threads_started":1}
I20260812 16:41:12.737778 4192 heartbeater.cc:347] Connected to a master server at 127.0.253.254:33455
I20260812 16:41:12.737883 4192 heartbeater.cc:464] Registering TS with master...
I20260812 16:41:12.737895 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.194:0
--local_ip_for_outbound_sockets=127.0.253.194
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:33455
--builtin_ntp_servers=127.0.253.212:40855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 16:41:12.738123 4192 heartbeater.cc:511] Master 127.0.253.254:33455 requested a full tablet report, sending...
I20260812 16:41:12.738821 3854 ts_manager.cc:194] Registered new tserver with Master: 6fe9845519d54d358d05947fd7c60c66 (127.0.253.193:33321)
I20260812 16:41:12.739811 3854 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.253.193:43627
W20260812 16:41:12.824958 4196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:12.825188 4196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:12.825212 4196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:12.826411 4196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:12.826457 4196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:12.826483 4196 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.194
I20260812 16:41:12.828689 4196 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:40855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:33455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.194
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:12.828963 4196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:12.829363 4196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:12.835565 4211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:12.835573 4204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:12.835628 4208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:12.836737 4196 server_base.cc:1034] running on GCE node
I20260812 16:41:12.836998 4196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:12.837342 4196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:12.838562 4196 hybrid_clock.cc:648] HybridClock initialized: now 1786552872838520 us; error 52 us; skew 500 ppm
I20260812 16:41:12.840323 4196 webserver.cc:460] Webserver started at http://127.0.253.194:46615/ using document root <none> and password file <none>
I20260812 16:41:12.840679 4196 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:12.840770 4196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:12.840939 4196 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:12.842739 4196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-1/data/instance:
uuid: "863fc2a309bd4be7b629c12c3a128a54"
format_stamp: "Formatted at 2026-08-12 16:41:12 on dist-test-slave-w4v5"
server_key: "8903afdaff7641c9d49821e045df614c"
server_key_iv: "86a111bc7a7deb5f6919e9f6f47c3392"
server_key_version: "encryptionkey@0"
I20260812 16:41:12.843279 4196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance:
uuid: "863fc2a309bd4be7b629c12c3a128a54"
format_stamp: "Formatted at 2026-08-12 16:41:12 on dist-test-slave-w4v5"
server_key: "8903afdaff7641c9d49821e045df614c"
server_key_iv: "86a111bc7a7deb5f6919e9f6f47c3392"
server_key_version: "encryptionkey@0"
I20260812 16:41:12.845412 4196 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 16:41:12.846345 4224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:12.846607 4196 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:41:12.846736 4196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-1/wal
uuid: "863fc2a309bd4be7b629c12c3a128a54"
format_stamp: "Formatted at 2026-08-12 16:41:12 on dist-test-slave-w4v5"
server_key: "8903afdaff7641c9d49821e045df614c"
server_key_iv: "86a111bc7a7deb5f6919e9f6f47c3392"
server_key_version: "encryptionkey@0"
I20260812 16:41:12.846845 4196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:12.861549 4196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:12.861850 4196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:12.861965 4196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:12.862175 4196 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:41:12.862564 4196 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:41:12.862608 4196 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:12.862633 4196 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:41:12.862687 4196 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:12.869513 4196 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.194:44009
I20260812 16:41:12.869544 4440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.194:44009 every 8 connection(s)
I20260812 16:41:12.869932 4196 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
I20260812 16:41:12.874367 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 4196
I20260812 16:41:12.874491 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance
I20260812 16:41:12.874737 1015 external_mini_cluster.cc:1428] Setting key a32985f0d55c6be3feb20bca6ff54b66
I20260812 16:41:12.875578 3922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:12.870780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:37295 (local address 127.0.253.254:33455)
0812 16:41:12.870867 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:12.870870 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:12.871063 (+ 193us) server_negotiation.cc:407] Connection header received
0812 16:41:12.871651 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:12.871653 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:12.871680 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:12.871769 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:12.872536 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:12.873036 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:12.874003 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:12.874194 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:12.874283 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:12.875311 (+ 1028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:12.875313 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:12.875313 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:41:12.875317 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:12.875333 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:41:12.875337 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:12.875378 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:12.875465 (+ 87us) server_negotiation.cc:299] Negotiation successful
0812 16:41:12.875493 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 16:41:12.875824 4445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:12.870714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:33455 (local address 127.0.253.194:37295)
0812 16:41:12.870928 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:12.870970 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:12.871523 (+ 553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:12.871875 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:12.871879 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:12.871887 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:41:12.872440 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:12.872445 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:12.873119 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:12.873120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:12.873926 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:12.873928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:12.874307 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:12.874308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:12.874441 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:12.875157 (+ 716us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:41:12.875158 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:41:12.875201 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:41:12.875228 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:41:12.875391 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:41:12.875395 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:41:12.875399 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:41:12.875462 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 16:41:12.875572 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":113,"threads_started":1}
I20260812 16:41:12.875914 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.195:0
--local_ip_for_outbound_sockets=127.0.253.195
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:33455
--builtin_ntp_servers=127.0.253.212:40855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 16:41:12.876747 4441 heartbeater.cc:347] Connected to a master server at 127.0.253.254:33455
I20260812 16:41:12.876855 4441 heartbeater.cc:464] Registering TS with master...
I20260812 16:41:12.877051 4441 heartbeater.cc:511] Master 127.0.253.254:33455 requested a full tablet report, sending...
I20260812 16:41:12.877410 3854 ts_manager.cc:194] Registered new tserver with Master: 863fc2a309bd4be7b629c12c3a128a54 (127.0.253.194:44009)
I20260812 16:41:12.877796 3854 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.253.194:37295
W20260812 16:41:12.960544 4446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:12.960754 4446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:12.960776 4446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:12.961843 4446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:12.961874 4446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:12.961898 4446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.195
I20260812 16:41:12.963241 4446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:40855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:33455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4446
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.195
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:12.963425 4446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:12.963671 4446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:12.968919 4452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:12.968933 4451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:12.968941 4455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:12.969169 4446 server_base.cc:1034] running on GCE node
I20260812 16:41:12.969398 4446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:12.969633 4446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:12.970801 4446 hybrid_clock.cc:648] HybridClock initialized: now 1786552872970780 us; error 38 us; skew 500 ppm
I20260812 16:41:12.972162 4446 webserver.cc:460] Webserver started at http://127.0.253.195:38961/ using document root <none> and password file <none>
I20260812 16:41:12.972407 4446 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:12.972476 4446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:12.972615 4446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:12.973665 4446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-2/data/instance:
uuid: "31a49ff5d35a4c52a0573cf9ee1f57be"
format_stamp: "Formatted at 2026-08-12 16:41:12 on dist-test-slave-w4v5"
server_key: "cb8f7c87aced72b6d0ab3e23f55cf96f"
server_key_iv: "1ba1bac4798be2c99236c5ccb3d6d8a6"
server_key_version: "encryptionkey@0"
I20260812 16:41:12.974040 4446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance:
uuid: "31a49ff5d35a4c52a0573cf9ee1f57be"
format_stamp: "Formatted at 2026-08-12 16:41:12 on dist-test-slave-w4v5"
server_key: "cb8f7c87aced72b6d0ab3e23f55cf96f"
server_key_iv: "1ba1bac4798be2c99236c5ccb3d6d8a6"
server_key_version: "encryptionkey@0"
I20260812 16:41:12.975586 4446 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:41:12.976312 4466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:12.976490 4446 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:41:12.976611 4446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-2/wal
uuid: "31a49ff5d35a4c52a0573cf9ee1f57be"
format_stamp: "Formatted at 2026-08-12 16:41:12 on dist-test-slave-w4v5"
server_key: "cb8f7c87aced72b6d0ab3e23f55cf96f"
server_key_iv: "1ba1bac4798be2c99236c5ccb3d6d8a6"
server_key_version: "encryptionkey@0"
I20260812 16:41:12.976701 4446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:12.994370 4446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:12.994678 4446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:12.994827 4446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:12.995055 4446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:41:12.995476 4446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:41:12.995532 4446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:12.995574 4446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:41:12.995600 4446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:13.001430 4446 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.195:43233
I20260812 16:41:13.001473 4682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.195:43233 every 8 connection(s)
I20260812 16:41:13.001853 4446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
I20260812 16:41:13.002290 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 4446
I20260812 16:41:13.002395 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance
I20260812 16:41:13.002624 1015 external_mini_cluster.cc:1428] Setting key e1a556ad86c7589cfa811409df76d345
I20260812 16:41:13.007437 3922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:13.002695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:47779 (local address 127.0.253.254:33455)
0812 16:41:13.002754 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:13.002757 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:13.003005 (+ 248us) server_negotiation.cc:407] Connection header received
0812 16:41:13.003711 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:13.003711 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:13.003721 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:13.003756 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:13.004395 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:13.004881 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:13.005876 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:13.006064 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:13.006119 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:13.007066 (+ 947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:13.007068 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:13.007068 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:41:13.007090 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:13.007105 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:41:13.007109 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:13.007172 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:13.007349 (+ 177us) server_negotiation.cc:299] Negotiation successful
0812 16:41:13.007378 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 16:41:13.007519 4685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:13.002653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:33455 (local address 127.0.253.195:47779)
0812 16:41:13.002885 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:13.002930 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:13.003629 (+ 699us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:13.003808 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:13.003811 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:13.003820 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:41:13.004325 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:13.004329 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:13.004970 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:13.004970 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:13.005795 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:13.005797 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:13.006145 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:13.006146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:13.006281 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:13.006904 (+ 623us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:41:13.006905 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:41:13.006934 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:41:13.006958 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:41:13.007175 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:41:13.007181 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:41:13.007183 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 16:41:13.007249 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 16:41:13.007332 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":90,"threads_started":1}
I20260812 16:41:13.008049 4683 heartbeater.cc:347] Connected to a master server at 127.0.253.254:33455
I20260812 16:41:13.008137 4683 heartbeater.cc:464] Registering TS with master...
I20260812 16:41:13.008284 4683 heartbeater.cc:511] Master 127.0.253.254:33455 requested a full tablet report, sending...
I20260812 16:41:13.008638 3854 ts_manager.cc:194] Registered new tserver with Master: 31a49ff5d35a4c52a0573cf9ee1f57be (127.0.253.195:43233)
I20260812 16:41:13.008976 3854 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.253.195:47779
I20260812 16:41:13.014405 1015 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:41:13.018884 3922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:13.015966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34574 (local address 127.0.253.254:33455)
0812 16:41:13.016061 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:13.016063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:13.016151 (+ 88us) server_negotiation.cc:407] Connection header received
0812 16:41:13.016175 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:13.016175 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:13.016186 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:13.016249 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:13.016410 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:13.016874 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:13.017691 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:13.017885 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:13.017953 (+ 68us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:13.018631 (+ 678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:13.018632 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:13.018633 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:41:13.018637 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:13.018651 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:41:13.018655 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:13.018739 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:13.018816 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 16:41:13.018826 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 16:41:13.020812 3854 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34574:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:41:13.022380 3854 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:41:13.029238 4701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:13.024804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:33321 (local address 127.0.0.1:35906)
0812 16:41:13.025202 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:13.025244 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:13.025330 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:13.025652 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:13.025655 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:13.025657 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:41:13.025739 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:13.025741 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:13.026740 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:13.026741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:13.028102 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:13.028105 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:13.028945 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:13.028946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:13.029024 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:13.029036 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 16:41:13.029112 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 16:41:13.029163 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":71,"threads_started":1}
I20260812 16:41:13.029255 4708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:13.025197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35906 (local address 127.0.253.193:33321)
0812 16:41:13.025382 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:13.025386 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:13.025399 (+ 13us) server_negotiation.cc:407] Connection header received
0812 16:41:13.025444 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:13.025447 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:13.025465 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:13.025560 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:41:13.025819 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:13.026637 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:13.028199 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:13.028848 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:13.028933 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:13.029043 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:13.029127 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 16:41:13.029163 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":79,"threads_started":1}
I20260812 16:41:13.029526 4696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:13.024909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56166 (local address 127.0.253.195:43233)
0812 16:41:13.025180 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:13.025186 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:13.025501 (+ 315us) server_negotiation.cc:407] Connection header received
0812 16:41:13.025578 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:13.025580 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:13.025599 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:13.025682 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:41:13.025915 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:13.027104 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:13.028375 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:13.029077 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:13.029136 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:13.029209 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:13.029400 (+ 191us) server_negotiation.cc:299] Negotiation successful
0812 16:41:13.029437 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":102,"threads_started":1}
I20260812 16:41:13.029471 4694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:13.025012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:44009 (local address 127.0.0.1:37268)
0812 16:41:13.025592 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:13.025600 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:13.025711 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:13.025925 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:13.025926 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:13.025928 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:41:13.025961 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:13.025962 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:13.027227 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:13.027228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:13.028087 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:13.028112 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:13.029230 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:13.029232 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:13.029299 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:13.029307 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 16:41:13.029358 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 16:41:13.029401 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"mutex_wait_us":2,"thread_start_us":109,"threads_started":1}
I20260812 16:41:13.029779 4695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:13.024805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.195:43233 (local address 127.0.0.1:56166)
0812 16:41:13.025424 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:13.025436 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:13.025503 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:13.025771 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:13.025771 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:13.025774 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:41:13.025819 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:13.025820 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:13.027181 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:13.027181 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:13.028313 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:13.028316 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:13.029238 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:13.029239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:13.029299 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:13.029305 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 16:41:13.029663 (+ 358us) client_negotiation.cc:240] Negotiation successful
0812 16:41:13.029712 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"mutex_wait_us":35,"thread_start_us":61,"threads_started":1}
I20260812 16:41:13.030012 4702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:13.025157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37268 (local address 127.0.253.194:44009)
0812 16:41:13.025350 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:13.025356 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:13.025683 (+ 327us) server_negotiation.cc:407] Connection header received
0812 16:41:13.025813 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:13.025815 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:13.025831 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:13.025913 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:41:13.026021 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:13.027165 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:13.028192 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:13.029012 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:13.029085 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:13.029189 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:13.029540 (+ 351us) server_negotiation.cc:299] Negotiation successful
0812 16:41:13.029924 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":101,"threads_started":1}
I20260812 16:41:13.030367 4074 tablet_service.cc:1467] Processing CreateTablet for tablet ad15e646652b4939a35e4e6b9d287c8b (DEFAULT_TABLE table=test-table [id=c84308abfe834eb3b6b258a8eb48402a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:41:13.030444 4561 tablet_service.cc:1467] Processing CreateTablet for tablet ad15e646652b4939a35e4e6b9d287c8b (DEFAULT_TABLE table=test-table [id=c84308abfe834eb3b6b258a8eb48402a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:41:13.030761 4074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad15e646652b4939a35e4e6b9d287c8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:13.030812 4561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad15e646652b4939a35e4e6b9d287c8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:13.033704 4709 tablet_bootstrap.cc:492] T ad15e646652b4939a35e4e6b9d287c8b P 31a49ff5d35a4c52a0573cf9ee1f57be: Bootstrap starting.
I20260812 16:41:13.033929 4710 tablet_bootstrap.cc:492] T ad15e646652b4939a35e4e6b9d287c8b P 6fe9845519d54d358d05947fd7c60c66: Bootstrap starting.
I20260812 16:41:13.034365 4709 tablet_bootstrap.cc:654] T ad15e646652b4939a35e4e6b9d287c8b P 31a49ff5d35a4c52a0573cf9ee1f57be: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:13.034329 4322 tablet_service.cc:1467] Processing CreateTablet for tablet ad15e646652b4939a35e4e6b9d287c8b (DEFAULT_TABLE table=test-table [id=c84308abfe834eb3b6b258a8eb48402a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:41:13.034657 4710 tablet_bootstrap.cc:654] T ad15e646652b4939a35e4e6b9d287c8b P 6fe9845519d54d358d05947fd7c60c66: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:13.034709 4322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad15e646652b4939a35e4e6b9d287c8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:13.034713 4709 log.cc:824] T ad15e646652b4939a35e4e6b9d287c8b P 31a49ff5d35a4c52a0573cf9ee1f57be: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:13.034973 4710 log.cc:824] T ad15e646652b4939a35e4e6b9d287c8b P 6fe9845519d54d358d05947fd7c60c66: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:13.035503 4709 tablet_bootstrap.cc:492] T ad15e646652b4939a35e4e6b9d287c8b P 31a49ff5d35a4c52a0573cf9ee1f57be: No bootstrap required, opened a new log
I20260812 16:41:13.035562 4709 ts_tablet_manager.cc:1397] T ad15e646652b4939a35e4e6b9d287c8b P 31a49ff5d35a4c52a0573cf9ee1f57be: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:41:13.037545 4719 tablet_bootstrap.cc:492] T ad15e646652b4939a35e4e6b9d287c8b P 863fc2a309bd4be7b629c12c3a128a54: Bootstrap starting.
I20260812 16:41:13.037626 4709 raft_consensus.cc:348] T ad15e646652b4939a35e4e6b9d287c8b P 31a49ff5d35a4c52a0573cf9ee1f57be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31a49ff5d35a4c52a0573cf9ee1f57be" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43233 } } peers { permanent_uuid: "6fe9845519d54d358d05947fd7c60c66" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 33321 } } peers { permanent_uuid: "863fc2a309bd4be7b629c12c3a128a54" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44009 } }
I20260812 16:41:13.037812 4709 raft_consensus.cc:374] T ad15e646652b4939a35e4e6b9d287c8b P 31a49ff5d35a4c52a0573cf9ee1f57be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:13.037861 4709 raft_consensus.cc:729] T ad15e646652b4939a35e4e6b9d287c8b P 31a49ff5d35a4c52a0573cf9ee1f57be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31a49ff5d35a4c52a0573cf9ee1f57be, State: Initialized, Role: FOLLOWER
I20260812 16:41:13.037978 4709 consensus_queue.cc:260] T ad15e646652b4939a35e4e6b9d287c8b P 31a49ff5d35a4c52a0573cf9ee1f57be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31a49ff5d35a4c52a0573cf9ee1f57be" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43233 } } peers { permanent_uuid: "6fe9845519d54d358d05947fd7c60c66" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 33321 } } peers { permanent_uuid: "863fc2a309bd4be7b629c12c3a128a54" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44009 } }
I20260812 16:41:13.038249 4709 ts_tablet_manager.cc:1428] T ad15e646652b4939a35e4e6b9d287c8b P 31a49ff5d35a4c52a0573cf9ee1f57be: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:41:13.038285 4719 tablet_bootstrap.cc:654] T ad15e646652b4939a35e4e6b9d287c8b P 863fc2a309bd4be7b629c12c3a128a54: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:13.038332 4683 heartbeater.cc:503] Master 127.0.253.254:33455 was elected leader, sending a full tablet report...
I20260812 16:41:13.038612 4719 log.cc:824] T ad15e646652b4939a35e4e6b9d287c8b P 863fc2a309bd4be7b629c12c3a128a54: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:13.038991 4710 tablet_bootstrap.cc:492] T ad15e646652b4939a35e4e6b9d287c8b P 6fe9845519d54d358d05947fd7c60c66: No bootstrap required, opened a new log
I20260812 16:41:13.039091 4710 ts_tablet_manager.cc:1397] T ad15e646652b4939a35e4e6b9d287c8b P 6fe9845519d54d358d05947fd7c60c66: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20260812 16:41:13.039444 4719 tablet_bootstrap.cc:492] T ad15e646652b4939a35e4e6b9d287c8b P 863fc2a309bd4be7b629c12c3a128a54: No bootstrap required, opened a new log
I20260812 16:41:13.039530 4719 ts_tablet_manager.cc:1397] T ad15e646652b4939a35e4e6b9d287c8b P 863fc2a309bd4be7b629c12c3a128a54: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:41:13.041249 4710 raft_consensus.cc:348] T ad15e646652b4939a35e4e6b9d287c8b P 6fe9845519d54d358d05947fd7c60c66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31a49ff5d35a4c52a0573cf9ee1f57be" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43233 } } peers { permanent_uuid: "6fe9845519d54d358d05947fd7c60c66" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 33321 } } peers { permanent_uuid: "863fc2a309bd4be7b629c12c3a128a54" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44009 } }
I20260812 16:41:13.041373 4710 raft_consensus.cc:374] T ad15e646652b4939a35e4e6b9d287c8b P 6fe9845519d54d358d05947fd7c60c66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:13.041396 4710 raft_consensus.cc:729] T ad15e646652b4939a35e4e6b9d287c8b P 6fe9845519d54d358d05947fd7c60c66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fe9845519d54d358d05947fd7c60c66, State: Initialized, Role: FOLLOWER
I20260812 16:41:13.041522 4710 consensus_queue.cc:260] T ad15e646652b4939a35e4e6b9d287c8b P 6fe9845519d54d358d05947fd7c60c66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31a49ff5d35a4c52a0573cf9ee1f57be" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43233 } } peers { permanent_uuid: "6fe9845519d54d358d05947fd7c60c66" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 33321 } } peers { permanent_uuid: "863fc2a309bd4be7b629c12c3a128a54" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44009 } }
I20260812 16:41:13.041518 4719 raft_consensus.cc:348] T ad15e646652b4939a35e4e6b9d287c8b P 863fc2a309bd4be7b629c12c3a128a54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31a49ff5d35a4c52a0573cf9ee1f57be" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43233 } } peers { permanent_uuid: "6fe9845519d54d358d05947fd7c60c66" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 33321 } } peers { permanent_uuid: "863fc2a309bd4be7b629c12c3a128a54" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44009 } }
I20260812 16:41:13.041679 4719 raft_consensus.cc:374] T ad15e646652b4939a35e4e6b9d287c8b P 863fc2a309bd4be7b629c12c3a128a54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:13.041731 4719 raft_consensus.cc:729] T ad15e646652b4939a35e4e6b9d287c8b P 863fc2a309bd4be7b629c12c3a128a54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 863fc2a309bd4be7b629c12c3a128a54, State: Initialized, Role: FOLLOWER
I20260812 16:41:13.041792 4710 ts_tablet_manager.cc:1428] T ad15e646652b4939a35e4e6b9d287c8b P 6fe9845519d54d358d05947fd7c60c66: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260812 16:41:13.041838 4192 heartbeater.cc:503] Master 127.0.253.254:33455 was elected leader, sending a full tablet report...
I20260812 16:41:13.042081 4719 consensus_queue.cc:260] T ad15e646652b4939a35e4e6b9d287c8b P 863fc2a309bd4be7b629c12c3a128a54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31a49ff5d35a4c52a0573cf9ee1f57be" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43233 } } peers { permanent_uuid: "6fe9845519d54d358d05947fd7c60c66" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 33321 } } peers { permanent_uuid: "863fc2a309bd4be7b629c12c3a128a54" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44009 } }
I20260812 16:41:13.042357 4719 ts_tablet_manager.cc:1428] T ad15e646652b4939a35e4e6b9d287c8b P 863fc2a309bd4be7b629c12c3a128a54: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:41:13.042461 4441 heartbeater.cc:503] Master 127.0.253.254:33455 was elected leader, sending a full tablet report...
I20260812 16:41:13.076885 4723 raft_consensus.cc:482] T ad15e646652b4939a35e4e6b9d287c8b P 6fe9845519d54d358d05947fd7c60c66 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:41:13.077056 4723 raft_consensus.cc:504] T ad15e646652b4939a35e4e6b9d287c8b P 6fe9845519d54d358d05947fd7c60c66 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31a49ff5d35a4c52a0573cf9ee1f57be" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43233 } } peers { permanent_uuid: "6fe9845519d54d358d05947fd7c60c66" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 33321 } } peers { permanent_uuid: "863fc2a309bd4be7b629c12c3a128a54" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44009 } }
I20260812 16:41:13.077510 4723 leader_election.cc:288] T ad15e646652b4939a35e4e6b9d287c8b P 6fe9845519d54d358d05947fd7c60c66 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31a49ff5d35a4c52a0573cf9ee1f57be (127.0.253.195:43233), 863fc2a309bd4be7b629c12c3a128a54 (127.0.253.194:44009)
I20260812 16:41:13.080708 4696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:13.077838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:35577 (local address 127.0.253.195:43233)
0812 16:41:13.077900 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:13.077903 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:13.078022 (+ 119us) server_negotiation.cc:407] Connection header received
0812 16:41:13.078122 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:13.078124 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:13.078140 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:13.078207 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:41:13.078376 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:13.078902 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:13.079947 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:13.080331 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:13.080418 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:13.080426 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:13.080587 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 16:41:13.080625 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 16:41:13.080711 4702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:13.077839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:34919 (local address 127.0.253.194:44009)
0812 16:41:13.077917 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:13.077920 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:13.077929 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:41:13.077968 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:13.077969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:13.077977 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:13.078043 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:41:13.078211 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:13.078700 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:13.079806 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:13.080346 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:13.080428 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:13.080437 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:13.080599 (+ 162us) server_negotiation.cc:299] Negotiation successful
0812 16:41:13.080639 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 16:41:13.080729 4195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:13.077717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:44009 (local address 127.0.253.193:34919)
0812 16:41:13.077784 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:13.077799 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:13.077863 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:13.078048 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:13.078049 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:13.078051 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:41:13.078129 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:13.078131 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:13.078787 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:13.078788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:13.079704 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:13.079707 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:13.080440 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:13.080441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:13.080525 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:13.080527 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 16:41:13.080593 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 16:41:13.080637 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 16:41:13.080852 4727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:13.077724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.195:43233 (local address 127.0.253.193:35577)
0812 16:41:13.077953 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:13.077961 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:13.078039 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:13.078223 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:13.078223 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:13.078225 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:41:13.078302 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:13.078303 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:13.078984 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:13.078985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:13.079856 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:13.079858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:13.080417 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:13.080418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:13.080512 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:13.080515 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:41:13.080583 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 16:41:13.080625 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":57,"thread_start_us":104,"threads_started":1}
I20260812 16:41:13.081202 4603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad15e646652b4939a35e4e6b9d287c8b" candidate_uuid: "6fe9845519d54d358d05947fd7c60c66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31a49ff5d35a4c52a0573cf9ee1f57be" is_pre_election: true
I20260812 16:41:13.081256 4363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad15e646652b4939a35e4e6b9d287c8b" candidate_uuid: "6fe9845519d54d358d05947fd7c60c66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "863fc2a309bd4be7b629c12c3a128a54" is_pre_election: true
I20260812 16:41:13.081362 4603 raft_consensus.cc:2454] T ad15e646652b4939a35e4e6b9d287c8b P 31a49ff5d35a4c52a0573cf9ee1f57be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fe9845519d54d358d05947fd7c60c66 in term 0.
I20260812 16:41:13.081429 4363 raft_consensus.cc:2454] T ad15e646652b4939a35e4e6b9d287c8b P 863fc2a309bd4be7b629c12c3a128a54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fe9845519d54d358d05947fd7c60c66 in term 0.
I20260812 16:41:13.081620 3982 leader_election.cc:302] T ad15e646652b4939a35e4e6b9d287c8b P 6fe9845519d54d358d05947fd7c60c66 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31a49ff5d35a4c52a0573cf9ee1f57be, 6fe9845519d54d358d05947fd7c60c66; no voters:
I20260812 16:41:13.081748 4723 raft_consensus.cc:2781] T ad15e646652b4939a35e4e6b9d287c8b P 6fe9845519d54d358d05947fd7c60c66 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:41:13.081815 4723 raft_consensus.cc:482] T ad15e646652b4939a35e4e6b9d287c8b P 6fe9845519d54d358d05947fd7c60c66 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:41:13.081853 4723 raft_consensus.cc:3037] T ad15e646652b4939a35e4e6b9d287c8b P 6fe9845519d54d358d05947fd7c60c66 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:13.082698 4723 raft_consensus.cc:504] T ad15e646652b4939a35e4e6b9d287c8b P 6fe9845519d54d358d05947fd7c60c66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31a49ff5d35a4c52a0573cf9ee1f57be" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43233 } } peers { permanent_uuid: "6fe9845519d54d358d05947fd7c60c66" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 33321 } } peers { permanent_uuid: "863fc2a309bd4be7b629c12c3a128a54" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44009 } }
I20260812 16:41:13.082844 4723 leader_election.cc:288] T ad15e646652b4939a35e4e6b9d287c8b P 6fe9845519d54d358d05947fd7c60c66 [CANDIDATE]: Term 1 election: Requested vote from peers 31a49ff5d35a4c52a0573cf9ee1f57be (127.0.253.195:43233), 863fc2a309bd4be7b629c12c3a128a54 (127.0.253.194:44009)
I20260812 16:41:13.082967 4603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad15e646652b4939a35e4e6b9d287c8b" candidate_uuid: "6fe9845519d54d358d05947fd7c60c66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31a49ff5d35a4c52a0573cf9ee1f57be"
I20260812 16:41:13.083033 4363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad15e646652b4939a35e4e6b9d287c8b" candidate_uuid: "6fe9845519d54d358d05947fd7c60c66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "863fc2a309bd4be7b629c12c3a128a54"
I20260812 16:41:13.083056 4603 raft_consensus.cc:3037] T ad15e646652b4939a35e4e6b9d287c8b P 31a49ff5d35a4c52a0573cf9ee1f57be [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:13.083168 4363 raft_consensus.cc:3037] T ad15e646652b4939a35e4e6b9d287c8b P 863fc2a309bd4be7b629c12c3a128a54 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:13.083839 4603 raft_consensus.cc:2454] T ad15e646652b4939a35e4e6b9d287c8b P 31a49ff5d35a4c52a0573cf9ee1f57be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe9845519d54d358d05947fd7c60c66 in term 1.
I20260812 16:41:13.083954 4363 raft_consensus.cc:2454] T ad15e646652b4939a35e4e6b9d287c8b P 863fc2a309bd4be7b629c12c3a128a54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe9845519d54d358d05947fd7c60c66 in term 1.
I20260812 16:41:13.084003 3982 leader_election.cc:302] T ad15e646652b4939a35e4e6b9d287c8b P 6fe9845519d54d358d05947fd7c60c66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31a49ff5d35a4c52a0573cf9ee1f57be, 6fe9845519d54d358d05947fd7c60c66; no voters:
I20260812 16:41:13.084098 4723 raft_consensus.cc:2781] T ad15e646652b4939a35e4e6b9d287c8b P 6fe9845519d54d358d05947fd7c60c66 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:41:13.084252 4723 raft_consensus.cc:686] T ad15e646652b4939a35e4e6b9d287c8b P 6fe9845519d54d358d05947fd7c60c66 [term 1 LEADER]: Becoming Leader. State: Replica: 6fe9845519d54d358d05947fd7c60c66, State: Running, Role: LEADER
I20260812 16:41:13.084354 4723 consensus_queue.cc:237] T ad15e646652b4939a35e4e6b9d287c8b P 6fe9845519d54d358d05947fd7c60c66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31a49ff5d35a4c52a0573cf9ee1f57be" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43233 } } peers { permanent_uuid: "6fe9845519d54d358d05947fd7c60c66" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 33321 } } peers { permanent_uuid: "863fc2a309bd4be7b629c12c3a128a54" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44009 } }
I20260812 16:41:13.085220 3854 catalog_manager.cc:5495] T ad15e646652b4939a35e4e6b9d287c8b P 6fe9845519d54d358d05947fd7c60c66 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fe9845519d54d358d05947fd7c60c66 (127.0.253.193). New cstate: current_term: 1 leader_uuid: "6fe9845519d54d358d05947fd7c60c66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31a49ff5d35a4c52a0573cf9ee1f57be" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fe9845519d54d358d05947fd7c60c66" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 33321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "863fc2a309bd4be7b629c12c3a128a54" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44009 } health_report { overall_health: UNKNOWN } } }
I20260812 16:41:13.102442 4708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:13.099906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35920 (local address 127.0.253.193:33321)
0812 16:41:13.099956 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:13.099959 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:13.099985 (+ 26us) server_negotiation.cc:407] Connection header received
0812 16:41:13.100101 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:13.100102 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:13.100122 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:13.100206 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:41:13.100355 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:13.100842 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:13.101636 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:13.101839 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:13.101918 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:13.102081 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:41:13.102170 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:41:13.102287 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:13.102321 (+ 34us) server_negotiation.cc:299] Negotiation successful
0812 16:41:13.102332 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 16:41:13.104358 4603 raft_consensus.cc:1261] T ad15e646652b4939a35e4e6b9d287c8b P 31a49ff5d35a4c52a0573cf9ee1f57be [term 1 FOLLOWER]: Refusing update from remote peer 6fe9845519d54d358d05947fd7c60c66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:41:13.104390 4363 raft_consensus.cc:1261] T ad15e646652b4939a35e4e6b9d287c8b P 863fc2a309bd4be7b629c12c3a128a54 [term 1 FOLLOWER]: Refusing update from remote peer 6fe9845519d54d358d05947fd7c60c66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:41:13.104753 4723 consensus_queue.cc:1035] T ad15e646652b4939a35e4e6b9d287c8b P 6fe9845519d54d358d05947fd7c60c66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "863fc2a309bd4be7b629c12c3a128a54" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 44009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:41:13.104892 4728 consensus_queue.cc:1035] T ad15e646652b4939a35e4e6b9d287c8b P 6fe9845519d54d358d05947fd7c60c66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31a49ff5d35a4c52a0573cf9ee1f57be" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 43233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:41:13.107583 4740 mvcc.cc:205] Tried to move back new op lower bound from 7317720568232722432 to 7317720568154112000. Current Snapshot: MvccSnapshot[applied={T|T < 7317720568232722432}]
I20260812 16:41:13.110153 3854 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:41:13.110265 3854 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:41:13.111181 3854 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 861949ed6896419380ae83d42161671d: Sending DeleteTablet for 3 replicas of tablet ad15e646652b4939a35e4e6b9d287c8b
I20260812 16:41:13.111533 4561 tablet_service.cc:1514] Processing DeleteTablet for tablet ad15e646652b4939a35e4e6b9d287c8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:41:13 UTC) from {username='slave'} at 127.0.0.1:56166
I20260812 16:41:13.111625 4322 tablet_service.cc:1514] Processing DeleteTablet for tablet ad15e646652b4939a35e4e6b9d287c8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:41:13 UTC) from {username='slave'} at 127.0.0.1:37268
I20260812 16:41:13.111805 4758 tablet_replica.cc:330] T ad15e646652b4939a35e4e6b9d287c8b P 31a49ff5d35a4c52a0573cf9ee1f57be: stopping tablet replica
I20260812 16:41:13.111851 4759 tablet_replica.cc:330] T ad15e646652b4939a35e4e6b9d287c8b P 863fc2a309bd4be7b629c12c3a128a54: stopping tablet replica
I20260812 16:41:13.111922 4758 raft_consensus.cc:2229] T ad15e646652b4939a35e4e6b9d287c8b P 31a49ff5d35a4c52a0573cf9ee1f57be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:41:13.111945 4759 raft_consensus.cc:2229] T ad15e646652b4939a35e4e6b9d287c8b P 863fc2a309bd4be7b629c12c3a128a54 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:41:13.112030 4758 raft_consensus.cc:2258] T ad15e646652b4939a35e4e6b9d287c8b P 31a49ff5d35a4c52a0573cf9ee1f57be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:41:13.112026 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 3953
I20260812 16:41:13.112046 4759 raft_consensus.cc:2258] T ad15e646652b4939a35e4e6b9d287c8b P 863fc2a309bd4be7b629c12c3a128a54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:41:13.113016 4759 ts_tablet_manager.cc:1905] T ad15e646652b4939a35e4e6b9d287c8b P 863fc2a309bd4be7b629c12c3a128a54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:41:13.115499 4759 ts_tablet_manager.cc:1918] T ad15e646652b4939a35e4e6b9d287c8b P 863fc2a309bd4be7b629c12c3a128a54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:41:13.115581 4759 log.cc:1196] T ad15e646652b4939a35e4e6b9d287c8b P 863fc2a309bd4be7b629c12c3a128a54: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-1/wal/wals/ad15e646652b4939a35e4e6b9d287c8b
I20260812 16:41:13.115883 4759 ts_tablet_manager.cc:1939] T ad15e646652b4939a35e4e6b9d287c8b P 863fc2a309bd4be7b629c12c3a128a54: Deleting consensus metadata
I20260812 16:41:13.116434 3826 catalog_manager.cc:4841] TS 863fc2a309bd4be7b629c12c3a128a54 (127.0.253.194:44009): tablet ad15e646652b4939a35e4e6b9d287c8b (table test-table [id=c84308abfe834eb3b6b258a8eb48402a]) successfully deleted
I20260812 16:41:13.123410 4758 ts_tablet_manager.cc:1905] T ad15e646652b4939a35e4e6b9d287c8b P 31a49ff5d35a4c52a0573cf9ee1f57be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:41:13.125226 4758 ts_tablet_manager.cc:1918] T ad15e646652b4939a35e4e6b9d287c8b P 31a49ff5d35a4c52a0573cf9ee1f57be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:41:13.125305 4758 log.cc:1196] T ad15e646652b4939a35e4e6b9d287c8b P 31a49ff5d35a4c52a0573cf9ee1f57be: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552619305671-1015-0/minicluster-data/ts-2/wal/wals/ad15e646652b4939a35e4e6b9d287c8b
I20260812 16:41:13.125636 4758 ts_tablet_manager.cc:1939] T ad15e646652b4939a35e4e6b9d287c8b P 31a49ff5d35a4c52a0573cf9ee1f57be: Deleting consensus metadata
I20260812 16:41:13.126149 3833 catalog_manager.cc:4841] TS 31a49ff5d35a4c52a0573cf9ee1f57be (127.0.253.195:43233): tablet ad15e646652b4939a35e4e6b9d287c8b (table test-table [id=c84308abfe834eb3b6b258a8eb48402a]) successfully deleted
W20260812 16:41:13.128970 3826 connection.cc:535] client connection to 127.0.253.193:33321 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 16:41:13.129072 3826 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 16:41:13.129201 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 4196
I20260812 16:41:13.132084 4695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:13.131925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:33321 (local address 127.0.0.1:35922)
0812 16:41:13.131980 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:13.132040 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.193:33321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260812 16:41:13.132207 3826 catalog_manager.cc:4584] TS 6fe9845519d54d358d05947fd7c60c66 (127.0.253.193:33321): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ad15e646652b4939a35e4e6b9d287c8b: Network error: Client connection negotiation failed: client connection to 127.0.253.193:33321: connect: Connection refused (error 111)
I20260812 16:41:13.139621 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 4446
I20260812 16:41:13.156924 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 3789
2026-08-12T16:41:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (774 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:41:13 dist-test-slave-w4v5 krb5kdc[4768](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:41:13 dist-test-slave-w4v5 krb5kdc[4768](info): set up 2 sockets
Aug 12 16:41:13 dist-test-slave-w4v5 krb5kdc[4768](info): commencing operation
krb5kdc: starting...
W20260812 16:41:14.503321 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.316s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:41:14 dist-test-slave-w4v5 krb5kdc[4768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552874, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:41:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:41:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
I20260812 16:41:14.630473 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:41451
--webserver_interface=127.0.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:44299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:41451
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:41:14.715318 4790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:14.715523 4790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:14.715546 4790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:14.716653 4790 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:41:14.716696 4790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:14.716710 4790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:14.716723 4790 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:41:14.716735 4790 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:41:14.718173 4790 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:44299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:41451
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:41451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:14.718380 4790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:14.718562 4790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:14.723521 4801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:14.723631 4806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:14.723650 4803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:14.723914 4790 server_base.cc:1034] running on GCE node
I20260812 16:41:14.724268 4790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:14.724519 4790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:14.725705 4790 hybrid_clock.cc:648] HybridClock initialized: now 1786552874725678 us; error 39 us; skew 500 ppm
Aug 12 16:41:14 dist-test-slave-w4v5 krb5kdc[4768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552874, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:41:14.728513 4790 init.cc:377] Logged in from keytab as kudu/127.0.253.254@KRBTEST.COM (short username kudu)
I20260812 16:41:14.728988 4790 webserver.cc:460] Webserver started at http://127.0.253.254:42859/ using document root <none> and password file <none>
I20260812 16:41:14.729246 4790 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:14.729318 4790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:14.729462 4790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:14.730561 4790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "d715bf70f4c148ff8f5c85181bc5351b"
format_stamp: "Formatted at 2026-08-12 16:41:14 on dist-test-slave-w4v5"
server_key: "e983dd4db99ce628070cc3e5e3a430c6"
server_key_iv: "4b5d8a08aceb564639aaf3362766ba08"
server_key_version: "encryptionkey@0"
I20260812 16:41:14.730939 4790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "d715bf70f4c148ff8f5c85181bc5351b"
format_stamp: "Formatted at 2026-08-12 16:41:14 on dist-test-slave-w4v5"
server_key: "e983dd4db99ce628070cc3e5e3a430c6"
server_key_iv: "4b5d8a08aceb564639aaf3362766ba08"
server_key_version: "encryptionkey@0"
I20260812 16:41:14.732414 4790 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:41:14.733464 4817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:14.733654 4790 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:41:14.733808 4790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "d715bf70f4c148ff8f5c85181bc5351b"
format_stamp: "Formatted at 2026-08-12 16:41:14 on dist-test-slave-w4v5"
server_key: "e983dd4db99ce628070cc3e5e3a430c6"
server_key_iv: "4b5d8a08aceb564639aaf3362766ba08"
server_key_version: "encryptionkey@0"
I20260812 16:41:14.733889 4790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:14.750703 4790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:14.752663 4790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:14.752808 4790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:14.756955 4790 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:41451
I20260812 16:41:14.757047 4918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:41451 every 8 connection(s)
I20260812 16:41:14.757462 4790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:41:14.758764 4919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:14.762128 4919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d715bf70f4c148ff8f5c85181bc5351b: Bootstrap starting.
I20260812 16:41:14.762872 4919 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d715bf70f4c148ff8f5c85181bc5351b: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:14.763232 4919 log.cc:824] T 00000000000000000000000000000000 P d715bf70f4c148ff8f5c85181bc5351b: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:14.764155 4919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d715bf70f4c148ff8f5c85181bc5351b: No bootstrap required, opened a new log
I20260812 16:41:14.765861 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 4790
I20260812 16:41:14.765975 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:41:14.766198 1015 external_mini_cluster.cc:1428] Setting key c3a9f76793b6cc022d26e9cfc98e1aec
I20260812 16:41:14.766481 4919 raft_consensus.cc:348] T 00000000000000000000000000000000 P d715bf70f4c148ff8f5c85181bc5351b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d715bf70f4c148ff8f5c85181bc5351b" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 41451 } }
I20260812 16:41:14.766671 4919 raft_consensus.cc:374] T 00000000000000000000000000000000 P d715bf70f4c148ff8f5c85181bc5351b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:14.766718 4919 raft_consensus.cc:729] T 00000000000000000000000000000000 P d715bf70f4c148ff8f5c85181bc5351b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d715bf70f4c148ff8f5c85181bc5351b, State: Initialized, Role: FOLLOWER
I20260812 16:41:14.766857 4919 consensus_queue.cc:260] T 00000000000000000000000000000000 P d715bf70f4c148ff8f5c85181bc5351b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d715bf70f4c148ff8f5c85181bc5351b" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 41451 } }
I20260812 16:41:14.766952 4919 raft_consensus.cc:388] T 00000000000000000000000000000000 P d715bf70f4c148ff8f5c85181bc5351b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:41:14.766995 4919 raft_consensus.cc:482] T 00000000000000000000000000000000 P d715bf70f4c148ff8f5c85181bc5351b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:41:14.767043 4919 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d715bf70f4c148ff8f5c85181bc5351b [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:14.767807 4919 raft_consensus.cc:504] T 00000000000000000000000000000000 P d715bf70f4c148ff8f5c85181bc5351b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d715bf70f4c148ff8f5c85181bc5351b" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 41451 } }
I20260812 16:41:14.767932 4919 leader_election.cc:302] T 00000000000000000000000000000000 P d715bf70f4c148ff8f5c85181bc5351b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d715bf70f4c148ff8f5c85181bc5351b; no voters:
I20260812 16:41:14.768122 4919 leader_election.cc:288] T 00000000000000000000000000000000 P d715bf70f4c148ff8f5c85181bc5351b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:41:14.768244 4925 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d715bf70f4c148ff8f5c85181bc5351b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:41:14.768435 4925 raft_consensus.cc:686] T 00000000000000000000000000000000 P d715bf70f4c148ff8f5c85181bc5351b [term 1 LEADER]: Becoming Leader. State: Replica: d715bf70f4c148ff8f5c85181bc5351b, State: Running, Role: LEADER
I20260812 16:41:14.768474 4919 sys_catalog.cc:564] T 00000000000000000000000000000000 P d715bf70f4c148ff8f5c85181bc5351b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:41:14.769897 4925 consensus_queue.cc:237] T 00000000000000000000000000000000 P d715bf70f4c148ff8f5c85181bc5351b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d715bf70f4c148ff8f5c85181bc5351b" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 41451 } }
I20260812 16:41:14.771224 4927 sys_catalog.cc:455] T 00000000000000000000000000000000 P d715bf70f4c148ff8f5c85181bc5351b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d715bf70f4c148ff8f5c85181bc5351b. Latest consensus state: current_term: 1 leader_uuid: "d715bf70f4c148ff8f5c85181bc5351b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d715bf70f4c148ff8f5c85181bc5351b" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 41451 } } }
I20260812 16:41:14.771219 4926 sys_catalog.cc:455] T 00000000000000000000000000000000 P d715bf70f4c148ff8f5c85181bc5351b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d715bf70f4c148ff8f5c85181bc5351b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d715bf70f4c148ff8f5c85181bc5351b" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 41451 } } }
I20260812 16:41:14.771332 4927 sys_catalog.cc:458] T 00000000000000000000000000000000 P d715bf70f4c148ff8f5c85181bc5351b [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:14.771338 4926 sys_catalog.cc:458] T 00000000000000000000000000000000 P d715bf70f4c148ff8f5c85181bc5351b [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:14.771577 4956 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:41:14.772212 4956 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 16:41:14 dist-test-slave-w4v5 krb5kdc[4768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552874, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:41:14.774372 4956 catalog_manager.cc:1347] Generated new cluster ID: f395d637b0f74ec2ac1f39e4137e9579
I20260812 16:41:14.774431 4956 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:41:14.778267 4924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:14.766778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60730 (local address 127.0.253.254:41451)
0812 16:41:14.766946 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:14.766956 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:14.766997 (+ 41us) server_negotiation.cc:407] Connection header received
0812 16:41:14.767154 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:14.767161 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:14.767217 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:14.767330 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:14.768201 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:14.769837 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:14.770897 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:14.771608 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:14.771739 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:14.774388 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:14.774392 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:14.774392 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:14.774405 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:14.776449 (+ 2044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:14.776824 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:14.776827 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:14.776827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:14.776935 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:14.777107 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:14.777107 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:14.777108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:14.777389 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:14.777530 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:14.777936 (+ 406us) server_negotiation.cc:299] Negotiation successful
0812 16:41:14.778066 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":92,"threads_started":1}
I20260812 16:41:14.787585 4956 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:41:14.788166 4956 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:41:14.792909 4956 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d715bf70f4c148ff8f5c85181bc5351b: Generated new TSK 0
I20260812 16:41:14.793092 4956 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab.
Entry for principal kudu/127.0.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab.
I20260812 16:41:14.824229 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.193:0
--local_ip_for_outbound_sockets=127.0.253.193
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:41451
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:44299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:41:14.909667 4962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:14.909871 4962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:14.909895 4962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:14.911150 4962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:14.911194 4962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:14.911221 4962 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.193
I20260812 16:41:14.912746 4962 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:44299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.193.keytab
--principal=kudu/127.0.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:41451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.193
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:14.913002 4962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:14.913283 4962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:14.918581 4975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:14.918593 4968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:14.918601 4972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:14.918941 4962 server_base.cc:1034] running on GCE node
I20260812 16:41:14.919185 4962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:14.919437 4962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:14.920608 4962 hybrid_clock.cc:648] HybridClock initialized: now 1786552874920603 us; error 41 us; skew 500 ppm
Aug 12 16:41:14 dist-test-slave-w4v5 krb5kdc[4768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552874, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:41:14.923373 4962 init.cc:377] Logged in from keytab as kudu/127.0.253.193@KRBTEST.COM (short username kudu)
I20260812 16:41:14.923947 4962 webserver.cc:460] Webserver started at http://127.0.253.193:34987/ using document root <none> and password file <none>
I20260812 16:41:14.924275 4962 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:14.924358 4962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:14.924525 4962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:14.925577 4962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-0/data/instance:
uuid: "23adc9635529498aa929af64e7f1c232"
format_stamp: "Formatted at 2026-08-12 16:41:14 on dist-test-slave-w4v5"
server_key: "fe8a8a03aff004690206053f9040abdc"
server_key_iv: "d9a5cdac1d8da8b14c58ef2bdb7de7de"
server_key_version: "encryptionkey@0"
I20260812 16:41:14.925951 4962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance:
uuid: "23adc9635529498aa929af64e7f1c232"
format_stamp: "Formatted at 2026-08-12 16:41:14 on dist-test-slave-w4v5"
server_key: "fe8a8a03aff004690206053f9040abdc"
server_key_iv: "d9a5cdac1d8da8b14c58ef2bdb7de7de"
server_key_version: "encryptionkey@0"
I20260812 16:41:14.927384 4962 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:41:14.928149 4991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:14.928328 4962 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:41:14.928436 4962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-0/wal
uuid: "23adc9635529498aa929af64e7f1c232"
format_stamp: "Formatted at 2026-08-12 16:41:14 on dist-test-slave-w4v5"
server_key: "fe8a8a03aff004690206053f9040abdc"
server_key_iv: "d9a5cdac1d8da8b14c58ef2bdb7de7de"
server_key_version: "encryptionkey@0"
I20260812 16:41:14.928511 4962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:14.940140 4962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:14.942240 4962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:14.942407 4962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:14.942643 4962 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:41:14.942962 4962 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:41:14.943019 4962 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:14.943063 4962 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:41:14.943133 4962 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:14.949038 4962 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.193:34089
I20260812 16:41:14.949083 5204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.193:34089 every 8 connection(s)
I20260812 16:41:14.949412 4962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
I20260812 16:41:14.950285 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 4962
I20260812 16:41:14.950409 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance
I20260812 16:41:14.950613 1015 external_mini_cluster.cc:1428] Setting key d4a0a02985da2e43282c2f15ba6a81f6
Aug 12 16:41:14 dist-test-slave-w4v5 krb5kdc[4768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552874, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.193@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:41:14.959303 4924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:14.950136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:55459 (local address 127.0.253.254:41451)
0812 16:41:14.950196 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:14.950198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:14.950396 (+ 198us) server_negotiation.cc:407] Connection header received
0812 16:41:14.951089 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:14.951090 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:14.951104 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:14.951151 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:14.952206 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:14.952716 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:14.953737 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:14.953942 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:14.954018 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:14.956234 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:14.956236 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:14.956236 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:14.956245 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:14.957801 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:14.958270 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:14.958272 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:14.958272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:14.958338 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:14.958600 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:14.958601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:14.958601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:14.958721 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:14.958796 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:14.959120 (+ 324us) server_negotiation.cc:299] Negotiation successful
0812 16:41:14.959212 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 16:41:14.959506 5208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:14.950072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41451 (local address 127.0.253.193:55459)
0812 16:41:14.950251 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:14.950293 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:14.950936 (+ 643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:14.951223 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:14.951227 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:14.951897 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:41:14.952104 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:14.952109 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:14.952816 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:14.952817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:14.953639 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:14.953641 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:14.954034 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:14.954035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:14.954177 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:14.954702 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:41:14.954711 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:41:14.956138 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:41:14.957916 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:14.957924 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:14.957926 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:14.958163 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:14.958408 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:14.958409 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:14.958409 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:14.958526 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:14.958797 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:41:14.958805 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:41:14.958949 (+ 144us) client_negotiation.cc:769] Sending connection context
0812 16:41:14.959053 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 16:41:14.959251 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":106,"threads_started":1}
WARNING: no policy specified for kudu/127.0.253.194@KRBTEST.COM; defaulting to no policy
I20260812 16:41:14.959975 5205 heartbeater.cc:347] Connected to a master server at 127.0.253.254:41451
I20260812 16:41:14.960071 5205 heartbeater.cc:464] Registering TS with master...
I20260812 16:41:14.960217 5205 heartbeater.cc:511] Master 127.0.253.254:41451 requested a full tablet report, sending...
I20260812 16:41:14.960764 4851 ts_manager.cc:194] Registered new tserver with Master: 23adc9635529498aa929af64e7f1c232 (127.0.253.193:34089)
I20260812 16:41:14.961323 4851 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.193@KRBTEST.COM'} at 127.0.253.193:55459
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab.
Entry for principal kudu/127.0.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab.
I20260812 16:41:14.978253 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.194:0
--local_ip_for_outbound_sockets=127.0.253.194
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:41451
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:44299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:41:15.062834 5213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:15.063055 5213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:15.063099 5213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:15.064339 5213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:15.064383 5213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:15.064409 5213 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.194
I20260812 16:41:15.065946 5213 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:44299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.194.keytab
--principal=kudu/127.0.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:41451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.194
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:15.066205 5213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:15.066449 5213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:15.072008 5220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:15.072022 5227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:15.072095 5223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:15.072304 5213 server_base.cc:1034] running on GCE node
I20260812 16:41:15.072520 5213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:15.072724 5213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:15.073889 5213 hybrid_clock.cc:648] HybridClock initialized: now 1786552875073869 us; error 30 us; skew 500 ppm
Aug 12 16:41:15 dist-test-slave-w4v5 krb5kdc[4768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552875, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:41:15.076721 5213 init.cc:377] Logged in from keytab as kudu/127.0.253.194@KRBTEST.COM (short username kudu)
I20260812 16:41:15.077219 5213 webserver.cc:460] Webserver started at http://127.0.253.194:43083/ using document root <none> and password file <none>
I20260812 16:41:15.077453 5213 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:15.077522 5213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:15.077690 5213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:15.078760 5213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-1/data/instance:
uuid: "5ae4288789874729ab8826fe255b9220"
format_stamp: "Formatted at 2026-08-12 16:41:15 on dist-test-slave-w4v5"
server_key: "2fd99a62bfa64c5880b9babc2247cb19"
server_key_iv: "2f3f088d9ddbe69b5a7e1842e2629042"
server_key_version: "encryptionkey@0"
I20260812 16:41:15.079281 5213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance:
uuid: "5ae4288789874729ab8826fe255b9220"
format_stamp: "Formatted at 2026-08-12 16:41:15 on dist-test-slave-w4v5"
server_key: "2fd99a62bfa64c5880b9babc2247cb19"
server_key_iv: "2f3f088d9ddbe69b5a7e1842e2629042"
server_key_version: "encryptionkey@0"
I20260812 16:41:15.080751 5213 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:41:15.081499 5237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:15.081734 5213 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:41:15.081843 5213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-1/wal
uuid: "5ae4288789874729ab8826fe255b9220"
format_stamp: "Formatted at 2026-08-12 16:41:15 on dist-test-slave-w4v5"
server_key: "2fd99a62bfa64c5880b9babc2247cb19"
server_key_iv: "2f3f088d9ddbe69b5a7e1842e2629042"
server_key_version: "encryptionkey@0"
I20260812 16:41:15.081933 5213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:15.090909 5213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:15.093375 5213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:15.093552 5213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:15.093806 5213 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:41:15.094177 5213 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:41:15.094230 5213 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:15.094269 5213 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:41:15.094300 5213 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:15.100504 5213 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.194:42791
I20260812 16:41:15.100564 5413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.194:42791 every 8 connection(s)
I20260812 16:41:15.100920 5213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
I20260812 16:41:15.104431 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 5213
I20260812 16:41:15.104532 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance
I20260812 16:41:15.104751 1015 external_mini_cluster.cc:1428] Setting key 05f3b048958c6672aa939096086de133
Aug 12 16:41:15 dist-test-slave-w4v5 krb5kdc[4768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552875, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.194@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:41:15.112371 4924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:15.101731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:55215 (local address 127.0.253.254:41451)
0812 16:41:15.101785 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:15.101787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:15.102137 (+ 350us) server_negotiation.cc:407] Connection header received
0812 16:41:15.102520 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:15.102521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:15.102533 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:15.102600 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:15.103646 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:15.104178 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:15.105669 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:15.105981 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:15.106063 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:15.108806 (+ 2743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:15.108808 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:15.108809 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:15.108819 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:15.110557 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:15.111119 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:15.111120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:15.111121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:15.111190 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:15.111473 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:15.111474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:15.111475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:15.111602 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:15.111697 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:15.112151 (+ 454us) server_negotiation.cc:299] Negotiation successful
0812 16:41:15.112268 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 16:41:15.112577 5421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:15.101680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41451 (local address 127.0.253.194:55215)
0812 16:41:15.101866 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:15.101910 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:15.102404 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:15.102689 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:15.102693 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:15.103320 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:41:15.103524 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:15.103530 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:15.104279 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:15.104279 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:15.105489 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:15.105494 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:15.106099 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:15.106100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:15.106242 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:15.106935 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:41:15.106945 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:41:15.108697 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:41:15.110749 (+ 2052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:15.110758 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:15.110761 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:15.111010 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:15.111283 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:15.111284 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:15.111285 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:15.111397 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:15.111691 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:41:15.111699 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:41:15.111962 (+ 263us) client_negotiation.cc:769] Sending connection context
0812 16:41:15.112095 (+ 133us) client_negotiation.cc:240] Negotiation successful
0812 16:41:15.112294 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":93,"threads_started":1}
I20260812 16:41:15.113457 5414 heartbeater.cc:347] Connected to a master server at 127.0.253.254:41451
I20260812 16:41:15.113590 5414 heartbeater.cc:464] Registering TS with master...
I20260812 16:41:15.113806 5414 heartbeater.cc:511] Master 127.0.253.254:41451 requested a full tablet report, sending...
I20260812 16:41:15.114270 4851 ts_manager.cc:194] Registered new tserver with Master: 5ae4288789874729ab8826fe255b9220 (127.0.253.194:42791)
I20260812 16:41:15.114806 4851 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.194@KRBTEST.COM'} at 127.0.253.194:55215
WARNING: no policy specified for kudu/127.0.253.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab.
Entry for principal kudu/127.0.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab.
I20260812 16:41:15.134331 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.195:0
--local_ip_for_outbound_sockets=127.0.253.195
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:41451
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:44299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:41:15.218274 5427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:15.218497 5427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:15.218521 5427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:15.219697 5427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:15.219754 5427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:15.219784 5427 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.195
I20260812 16:41:15.221210 5427 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:44299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.195.keytab
--principal=kudu/127.0.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:41451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.195
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:15.221411 5427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:15.221588 5427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:15.226734 5438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:15.226711 5436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:15.226794 5445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:15.226960 5427 server_base.cc:1034] running on GCE node
I20260812 16:41:15.227191 5427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:15.227396 5427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:15.228536 5427 hybrid_clock.cc:648] HybridClock initialized: now 1786552875228495 us; error 59 us; skew 500 ppm
Aug 12 16:41:15 dist-test-slave-w4v5 krb5kdc[4768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552875, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:41:15.231050 5427 init.cc:377] Logged in from keytab as kudu/127.0.253.195@KRBTEST.COM (short username kudu)
I20260812 16:41:15.231699 5427 webserver.cc:460] Webserver started at http://127.0.253.195:34137/ using document root <none> and password file <none>
I20260812 16:41:15.231959 5427 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:15.232030 5427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:15.232174 5427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:15.233314 5427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-2/data/instance:
uuid: "a55f941fc2d440afa7abf393c0ed4d63"
format_stamp: "Formatted at 2026-08-12 16:41:15 on dist-test-slave-w4v5"
server_key: "be2558945a25267b1a6dad75f3f5716d"
server_key_iv: "5fe4f7288ecea6a846ffad568b0d05e3"
server_key_version: "encryptionkey@0"
I20260812 16:41:15.233722 5427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance:
uuid: "a55f941fc2d440afa7abf393c0ed4d63"
format_stamp: "Formatted at 2026-08-12 16:41:15 on dist-test-slave-w4v5"
server_key: "be2558945a25267b1a6dad75f3f5716d"
server_key_iv: "5fe4f7288ecea6a846ffad568b0d05e3"
server_key_version: "encryptionkey@0"
I20260812 16:41:15.235142 5427 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260812 16:41:15.235934 5472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:15.236133 5427 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:41:15.236240 5427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-2/wal
uuid: "a55f941fc2d440afa7abf393c0ed4d63"
format_stamp: "Formatted at 2026-08-12 16:41:15 on dist-test-slave-w4v5"
server_key: "be2558945a25267b1a6dad75f3f5716d"
server_key_iv: "5fe4f7288ecea6a846ffad568b0d05e3"
server_key_version: "encryptionkey@0"
I20260812 16:41:15.236326 5427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:15.256042 5427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:15.258185 5427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:15.258368 5427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:15.258635 5427 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:41:15.258989 5427 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:41:15.259047 5427 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:15.259081 5427 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:41:15.259158 5427 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:15.265378 5427 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.195:34211
I20260812 16:41:15.265432 5664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.195:34211 every 8 connection(s)
I20260812 16:41:15.265808 5427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
I20260812 16:41:15.270627 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 5427
I20260812 16:41:15.270768 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance
I20260812 16:41:15.270975 1015 external_mini_cluster.cc:1428] Setting key 940f72be700f0c513047875fd9df5b47
Aug 12 16:41:15 dist-test-slave-w4v5 krb5kdc[4768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552875, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.195@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:41:15.276796 4924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:15.266624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:42823 (local address 127.0.253.254:41451)
0812 16:41:15.266726 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:15.266728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:15.266951 (+ 223us) server_negotiation.cc:407] Connection header received
0812 16:41:15.267710 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:15.267712 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:15.267725 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:15.267784 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:15.268999 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:15.269599 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:15.270720 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:15.271044 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:15.271158 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:15.273734 (+ 2576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:15.273735 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:15.273735 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:15.273742 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:15.275346 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:15.275739 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:15.275740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:15.275741 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:15.275795 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:15.276134 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:15.276135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:15.276136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:15.276262 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:15.276324 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:15.276638 (+ 314us) server_negotiation.cc:299] Negotiation successful
0812 16:41:15.276731 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 16:41:15.276954 5670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:15.266593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:41451 (local address 127.0.253.195:42823)
0812 16:41:15.266783 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:15.266843 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:15.267577 (+ 734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:15.267840 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:15.267843 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:15.268599 (+ 756us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:41:15.268888 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:15.268893 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:15.269702 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:15.269703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:15.270599 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:15.270602 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:15.271175 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:15.271176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:15.271355 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:15.272086 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:41:15.272097 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:41:15.273621 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:41:15.275423 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:15.275430 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:15.275432 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:15.275669 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:15.275886 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:15.275887 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:15.275888 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:15.276035 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:15.276336 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:41:15.276342 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:41:15.276474 (+ 132us) client_negotiation.cc:769] Sending connection context
0812 16:41:15.276571 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 16:41:15.276721 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":85,"threads_started":1}
I20260812 16:41:15.277601 5665 heartbeater.cc:347] Connected to a master server at 127.0.253.254:41451
I20260812 16:41:15.277719 5665 heartbeater.cc:464] Registering TS with master...
I20260812 16:41:15.277922 5665 heartbeater.cc:511] Master 127.0.253.254:41451 requested a full tablet report, sending...
I20260812 16:41:15.278419 4851 ts_manager.cc:194] Registered new tserver with Master: a55f941fc2d440afa7abf393c0ed4d63 (127.0.253.195:34211)
I20260812 16:41:15.278784 4851 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.253.195@KRBTEST.COM'} at 127.0.253.195:42823
I20260812 16:41:15.283576 1015 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:41:15.292073 4924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:15.285188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60746 (local address 127.0.253.254:41451)
0812 16:41:15.285237 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:15.285239 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:15.285329 (+ 90us) server_negotiation.cc:407] Connection header received
0812 16:41:15.285389 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:15.285390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:15.285402 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:15.285457 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:15.286151 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:15.286697 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:15.287689 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:15.287907 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:15.287964 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:15.289302 (+ 1338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:15.289303 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:15.289304 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:15.289311 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:15.290832 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:15.291335 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:15.291336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:15.291337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:15.291393 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:15.291609 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:15.291609 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:15.291609 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:15.291749 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:15.291825 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:15.291942 (+ 117us) server_negotiation.cc:299] Negotiation successful
0812 16:41:15.292006 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 16:41:15.294238 4851 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60746:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:41:15.295944 4851 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:41:15.303143 5691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:15.298588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42460 (local address 127.0.253.193:34089)
0812 16:41:15.299520 (+ 932us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:15.299524 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:15.299534 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:41:15.299575 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:15.299577 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:15.299591 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:15.299658 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:41:15.299858 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:15.300558 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:15.302214 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:15.302751 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:15.302823 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:15.302878 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:15.302989 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 16:41:15.303031 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":890,"thread_start_us":136,"threads_started":1}
I20260812 16:41:15.303140 5690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:15.298470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:34089 (local address 127.0.0.1:42460)
0812 16:41:15.298837 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:15.298882 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:15.298955 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:15.299710 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:15.299710 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:15.299713 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:41:15.299770 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:15.299771 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:15.300678 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:15.300679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:15.302112 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:15.302114 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:15.302828 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:15.302829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:15.302897 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:15.302903 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 16:41:15.302961 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 16:41:15.303009 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":74,"thread_start_us":54,"threads_started":1}
I20260812 16:41:15.303244 5694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:15.298622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58256 (local address 127.0.253.194:42791)
0812 16:41:15.299042 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:15.299046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:15.299056 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:41:15.299113 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:15.299116 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:15.299132 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:15.299215 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:41:15.299420 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:15.300599 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:15.301937 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:15.302620 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:15.302699 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:15.302762 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:15.303063 (+ 301us) server_negotiation.cc:299] Negotiation successful
0812 16:41:15.303147 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":105,"threads_started":1}
I20260812 16:41:15.303321 5698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:15.298676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.195:34211 (local address 127.0.0.1:47834)
0812 16:41:15.299320 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:15.299331 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:15.299379 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:15.299576 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:15.299578 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:15.299580 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:41:15.299624 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:15.299626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:15.300843 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:15.300844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:15.302101 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:15.302104 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:15.303061 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:15.303062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:15.303143 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:15.303148 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:41:15.303201 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 16:41:15.303247 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"mutex_wait_us":17,"thread_start_us":89,"threads_started":1}
I20260812 16:41:15.303414 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:15.298502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:42791 (local address 127.0.0.1:58256)
0812 16:41:15.298834 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:15.298880 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:15.298959 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:15.299234 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:15.299237 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:15.299240 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:41:15.299316 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:15.299318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:15.300726 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:15.300728 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:15.301839 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:15.301842 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:15.302720 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:15.302722 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:15.302822 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:15.302838 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 16:41:15.302908 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 16:41:15.302972 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"mutex_wait_us":68,"thread_start_us":80,"threads_started":1}
I20260812 16:41:15.303773 5699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:15.298806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47834 (local address 127.0.253.195:34211)
0812 16:41:15.299018 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:15.299024 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:15.299376 (+ 352us) server_negotiation.cc:407] Connection header received
0812 16:41:15.299458 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:15.299462 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:15.299478 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:15.299560 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:41:15.299717 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:15.300743 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:15.302198 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:15.302898 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:15.302980 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:15.303032 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:15.303643 (+ 611us) server_negotiation.cc:299] Negotiation successful
0812 16:41:15.303692 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":103,"threads_started":1}
I20260812 16:41:15.304235 5312 tablet_service.cc:1467] Processing CreateTablet for tablet 74776322eae74e7aa7794747ab1a85c0 (DEFAULT_TABLE table=test-table [id=607bce51c8ed4ed4a475051cdcec2a59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:41:15.304338 5545 tablet_service.cc:1467] Processing CreateTablet for tablet 74776322eae74e7aa7794747ab1a85c0 (DEFAULT_TABLE table=test-table [id=607bce51c8ed4ed4a475051cdcec2a59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:41:15.304510 5070 tablet_service.cc:1467] Processing CreateTablet for tablet 74776322eae74e7aa7794747ab1a85c0 (DEFAULT_TABLE table=test-table [id=607bce51c8ed4ed4a475051cdcec2a59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:41:15.304648 5312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74776322eae74e7aa7794747ab1a85c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:15.304648 5545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74776322eae74e7aa7794747ab1a85c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:15.304872 5070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74776322eae74e7aa7794747ab1a85c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:15.307593 5700 tablet_bootstrap.cc:492] T 74776322eae74e7aa7794747ab1a85c0 P 23adc9635529498aa929af64e7f1c232: Bootstrap starting.
I20260812 16:41:15.308113 5702 tablet_bootstrap.cc:492] T 74776322eae74e7aa7794747ab1a85c0 P 5ae4288789874729ab8826fe255b9220: Bootstrap starting.
I20260812 16:41:15.308481 5700 tablet_bootstrap.cc:654] T 74776322eae74e7aa7794747ab1a85c0 P 23adc9635529498aa929af64e7f1c232: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:15.308825 5702 tablet_bootstrap.cc:654] T 74776322eae74e7aa7794747ab1a85c0 P 5ae4288789874729ab8826fe255b9220: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:15.309115 5700 log.cc:824] T 74776322eae74e7aa7794747ab1a85c0 P 23adc9635529498aa929af64e7f1c232: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:15.309442 5702 log.cc:824] T 74776322eae74e7aa7794747ab1a85c0 P 5ae4288789874729ab8826fe255b9220: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:15.309830 5701 tablet_bootstrap.cc:492] T 74776322eae74e7aa7794747ab1a85c0 P a55f941fc2d440afa7abf393c0ed4d63: Bootstrap starting.
I20260812 16:41:15.310039 5700 tablet_bootstrap.cc:492] T 74776322eae74e7aa7794747ab1a85c0 P 23adc9635529498aa929af64e7f1c232: No bootstrap required, opened a new log
I20260812 16:41:15.310140 5700 ts_tablet_manager.cc:1397] T 74776322eae74e7aa7794747ab1a85c0 P 23adc9635529498aa929af64e7f1c232: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 16:41:15.310256 5702 tablet_bootstrap.cc:492] T 74776322eae74e7aa7794747ab1a85c0 P 5ae4288789874729ab8826fe255b9220: No bootstrap required, opened a new log
I20260812 16:41:15.310328 5702 ts_tablet_manager.cc:1397] T 74776322eae74e7aa7794747ab1a85c0 P 5ae4288789874729ab8826fe255b9220: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:41:15.310523 5701 tablet_bootstrap.cc:654] T 74776322eae74e7aa7794747ab1a85c0 P a55f941fc2d440afa7abf393c0ed4d63: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:15.311125 5701 log.cc:824] T 74776322eae74e7aa7794747ab1a85c0 P a55f941fc2d440afa7abf393c0ed4d63: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:15.312064 5700 raft_consensus.cc:348] T 74776322eae74e7aa7794747ab1a85c0 P 23adc9635529498aa929af64e7f1c232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23adc9635529498aa929af64e7f1c232" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 34089 } } peers { permanent_uuid: "a55f941fc2d440afa7abf393c0ed4d63" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 34211 } } peers { permanent_uuid: "5ae4288789874729ab8826fe255b9220" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42791 } }
I20260812 16:41:15.312225 5701 tablet_bootstrap.cc:492] T 74776322eae74e7aa7794747ab1a85c0 P a55f941fc2d440afa7abf393c0ed4d63: No bootstrap required, opened a new log
I20260812 16:41:15.312253 5700 raft_consensus.cc:374] T 74776322eae74e7aa7794747ab1a85c0 P 23adc9635529498aa929af64e7f1c232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:15.312292 5701 ts_tablet_manager.cc:1397] T 74776322eae74e7aa7794747ab1a85c0 P a55f941fc2d440afa7abf393c0ed4d63: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 16:41:15.312302 5700 raft_consensus.cc:729] T 74776322eae74e7aa7794747ab1a85c0 P 23adc9635529498aa929af64e7f1c232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23adc9635529498aa929af64e7f1c232, State: Initialized, Role: FOLLOWER
I20260812 16:41:15.312444 5700 consensus_queue.cc:260] T 74776322eae74e7aa7794747ab1a85c0 P 23adc9635529498aa929af64e7f1c232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23adc9635529498aa929af64e7f1c232" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 34089 } } peers { permanent_uuid: "a55f941fc2d440afa7abf393c0ed4d63" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 34211 } } peers { permanent_uuid: "5ae4288789874729ab8826fe255b9220" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42791 } }
I20260812 16:41:15.312577 5702 raft_consensus.cc:348] T 74776322eae74e7aa7794747ab1a85c0 P 5ae4288789874729ab8826fe255b9220 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23adc9635529498aa929af64e7f1c232" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 34089 } } peers { permanent_uuid: "a55f941fc2d440afa7abf393c0ed4d63" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 34211 } } peers { permanent_uuid: "5ae4288789874729ab8826fe255b9220" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42791 } }
I20260812 16:41:15.312752 5702 raft_consensus.cc:374] T 74776322eae74e7aa7794747ab1a85c0 P 5ae4288789874729ab8826fe255b9220 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:15.312772 5700 ts_tablet_manager.cc:1428] T 74776322eae74e7aa7794747ab1a85c0 P 23adc9635529498aa929af64e7f1c232: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:41:15.312799 5702 raft_consensus.cc:729] T 74776322eae74e7aa7794747ab1a85c0 P 5ae4288789874729ab8826fe255b9220 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ae4288789874729ab8826fe255b9220, State: Initialized, Role: FOLLOWER
I20260812 16:41:15.312852 5205 heartbeater.cc:503] Master 127.0.253.254:41451 was elected leader, sending a full tablet report...
I20260812 16:41:15.312932 5702 consensus_queue.cc:260] T 74776322eae74e7aa7794747ab1a85c0 P 5ae4288789874729ab8826fe255b9220 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23adc9635529498aa929af64e7f1c232" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 34089 } } peers { permanent_uuid: "a55f941fc2d440afa7abf393c0ed4d63" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 34211 } } peers { permanent_uuid: "5ae4288789874729ab8826fe255b9220" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42791 } }
I20260812 16:41:15.313211 5702 ts_tablet_manager.cc:1428] T 74776322eae74e7aa7794747ab1a85c0 P 5ae4288789874729ab8826fe255b9220: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:41:15.313613 5414 heartbeater.cc:503] Master 127.0.253.254:41451 was elected leader, sending a full tablet report...
I20260812 16:41:15.314227 5701 raft_consensus.cc:348] T 74776322eae74e7aa7794747ab1a85c0 P a55f941fc2d440afa7abf393c0ed4d63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23adc9635529498aa929af64e7f1c232" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 34089 } } peers { permanent_uuid: "a55f941fc2d440afa7abf393c0ed4d63" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 34211 } } peers { permanent_uuid: "5ae4288789874729ab8826fe255b9220" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42791 } }
I20260812 16:41:15.314396 5701 raft_consensus.cc:374] T 74776322eae74e7aa7794747ab1a85c0 P a55f941fc2d440afa7abf393c0ed4d63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:15.314421 5701 raft_consensus.cc:729] T 74776322eae74e7aa7794747ab1a85c0 P a55f941fc2d440afa7abf393c0ed4d63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a55f941fc2d440afa7abf393c0ed4d63, State: Initialized, Role: FOLLOWER
I20260812 16:41:15.314503 5701 consensus_queue.cc:260] T 74776322eae74e7aa7794747ab1a85c0 P a55f941fc2d440afa7abf393c0ed4d63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23adc9635529498aa929af64e7f1c232" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 34089 } } peers { permanent_uuid: "a55f941fc2d440afa7abf393c0ed4d63" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 34211 } } peers { permanent_uuid: "5ae4288789874729ab8826fe255b9220" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42791 } }
I20260812 16:41:15.314703 5701 ts_tablet_manager.cc:1428] T 74776322eae74e7aa7794747ab1a85c0 P a55f941fc2d440afa7abf393c0ed4d63: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260812 16:41:15.314754 5665 heartbeater.cc:503] Master 127.0.253.254:41451 was elected leader, sending a full tablet report...
W20260812 16:41:15.351580 5416 tablet.cc:2369] T 74776322eae74e7aa7794747ab1a85c0 P 5ae4288789874729ab8826fe255b9220: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 16:41:15.450171 5207 tablet.cc:2369] T 74776322eae74e7aa7794747ab1a85c0 P 23adc9635529498aa929af64e7f1c232: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 16:41:15.516422 5667 tablet.cc:2369] T 74776322eae74e7aa7794747ab1a85c0 P a55f941fc2d440afa7abf393c0ed4d63: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:41:15.534495 5715 raft_consensus.cc:482] T 74776322eae74e7aa7794747ab1a85c0 P a55f941fc2d440afa7abf393c0ed4d63 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:41:15.534652 5715 raft_consensus.cc:504] T 74776322eae74e7aa7794747ab1a85c0 P a55f941fc2d440afa7abf393c0ed4d63 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23adc9635529498aa929af64e7f1c232" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 34089 } } peers { permanent_uuid: "a55f941fc2d440afa7abf393c0ed4d63" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 34211 } } peers { permanent_uuid: "5ae4288789874729ab8826fe255b9220" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42791 } }
I20260812 16:41:15.534991 5715 leader_election.cc:288] T 74776322eae74e7aa7794747ab1a85c0 P a55f941fc2d440afa7abf393c0ed4d63 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23adc9635529498aa929af64e7f1c232 (127.0.253.193:34089), 5ae4288789874729ab8826fe255b9220 (127.0.253.194:42791)
I20260812 16:41:15.538477 5670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:15.535264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:34089 (local address 127.0.253.195:41211)
0812 16:41:15.535312 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:15.535326 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:15.535374 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:15.535631 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:15.535632 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:15.535635 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:41:15.535785 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:15.535788 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:15.536600 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:15.536601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:15.537484 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:15.537487 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:15.538204 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:15.538205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:15.538292 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:15.538297 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:41:15.538352 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 16:41:15.538407 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260812 16:41:15.538540 5691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:15.535409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:41211 (local address 127.0.253.193:34089)
0812 16:41:15.535470 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:15.535472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:15.535482 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:41:15.535517 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:15.535518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:15.535529 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:15.535622 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:41:15.535898 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:15.536501 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:15.537576 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:15.538093 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:15.538197 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:15.538206 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:15.538394 (+ 188us) server_negotiation.cc:299] Negotiation successful
0812 16:41:15.538436 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 16:41:15.538693 5694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:15.535338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:35329 (local address 127.0.253.194:42791)
0812 16:41:15.535423 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:15.535426 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:15.535776 (+ 350us) server_negotiation.cc:407] Connection header received
0812 16:41:15.535807 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:15.535808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:15.535822 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:15.535878 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:41:15.536052 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:15.536523 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:15.537809 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:15.538271 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:15.538347 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:15.538356 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:15.538547 (+ 191us) server_negotiation.cc:299] Negotiation successful
0812 16:41:15.538611 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 16:41:15.538710 5717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:15.535264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:42791 (local address 127.0.253.195:35329)
0812 16:41:15.535630 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:15.535681 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:15.535737 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:15.535899 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:15.535900 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:15.535902 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:41:15.535979 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:15.535980 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:15.536633 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:15.536635 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:15.537675 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:15.537678 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:15.538375 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:15.538376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:15.538459 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:15.538463 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:41:15.538529 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 16:41:15.538575 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"mutex_wait_us":170,"thread_start_us":137,"threads_started":1}
I20260812 16:41:15.538920 5104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74776322eae74e7aa7794747ab1a85c0" candidate_uuid: "a55f941fc2d440afa7abf393c0ed4d63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23adc9635529498aa929af64e7f1c232" is_pre_election: true
I20260812 16:41:15.539047 5346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74776322eae74e7aa7794747ab1a85c0" candidate_uuid: "a55f941fc2d440afa7abf393c0ed4d63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ae4288789874729ab8826fe255b9220" is_pre_election: true
I20260812 16:41:15.539126 5104 raft_consensus.cc:2454] T 74776322eae74e7aa7794747ab1a85c0 P 23adc9635529498aa929af64e7f1c232 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a55f941fc2d440afa7abf393c0ed4d63 in term 0.
I20260812 16:41:15.539245 5346 raft_consensus.cc:2454] T 74776322eae74e7aa7794747ab1a85c0 P 5ae4288789874729ab8826fe255b9220 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a55f941fc2d440afa7abf393c0ed4d63 in term 0.
I20260812 16:41:15.539412 5475 leader_election.cc:302] T 74776322eae74e7aa7794747ab1a85c0 P a55f941fc2d440afa7abf393c0ed4d63 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23adc9635529498aa929af64e7f1c232, a55f941fc2d440afa7abf393c0ed4d63; no voters:
I20260812 16:41:15.539613 5715 raft_consensus.cc:2781] T 74776322eae74e7aa7794747ab1a85c0 P a55f941fc2d440afa7abf393c0ed4d63 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:41:15.539667 5715 raft_consensus.cc:482] T 74776322eae74e7aa7794747ab1a85c0 P a55f941fc2d440afa7abf393c0ed4d63 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:41:15.539708 5715 raft_consensus.cc:3037] T 74776322eae74e7aa7794747ab1a85c0 P a55f941fc2d440afa7abf393c0ed4d63 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:15.540493 5715 raft_consensus.cc:504] T 74776322eae74e7aa7794747ab1a85c0 P a55f941fc2d440afa7abf393c0ed4d63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23adc9635529498aa929af64e7f1c232" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 34089 } } peers { permanent_uuid: "a55f941fc2d440afa7abf393c0ed4d63" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 34211 } } peers { permanent_uuid: "5ae4288789874729ab8826fe255b9220" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42791 } }
I20260812 16:41:15.540745 5715 leader_election.cc:288] T 74776322eae74e7aa7794747ab1a85c0 P a55f941fc2d440afa7abf393c0ed4d63 [CANDIDATE]: Term 1 election: Requested vote from peers 23adc9635529498aa929af64e7f1c232 (127.0.253.193:34089), 5ae4288789874729ab8826fe255b9220 (127.0.253.194:42791)
I20260812 16:41:15.540907 5104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74776322eae74e7aa7794747ab1a85c0" candidate_uuid: "a55f941fc2d440afa7abf393c0ed4d63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23adc9635529498aa929af64e7f1c232"
I20260812 16:41:15.540932 5346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74776322eae74e7aa7794747ab1a85c0" candidate_uuid: "a55f941fc2d440afa7abf393c0ed4d63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ae4288789874729ab8826fe255b9220"
I20260812 16:41:15.540992 5104 raft_consensus.cc:3037] T 74776322eae74e7aa7794747ab1a85c0 P 23adc9635529498aa929af64e7f1c232 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:15.541039 5346 raft_consensus.cc:3037] T 74776322eae74e7aa7794747ab1a85c0 P 5ae4288789874729ab8826fe255b9220 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:15.541826 5346 raft_consensus.cc:2454] T 74776322eae74e7aa7794747ab1a85c0 P 5ae4288789874729ab8826fe255b9220 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a55f941fc2d440afa7abf393c0ed4d63 in term 1.
I20260812 16:41:15.541826 5104 raft_consensus.cc:2454] T 74776322eae74e7aa7794747ab1a85c0 P 23adc9635529498aa929af64e7f1c232 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a55f941fc2d440afa7abf393c0ed4d63 in term 1.
I20260812 16:41:15.542057 5478 leader_election.cc:302] T 74776322eae74e7aa7794747ab1a85c0 P a55f941fc2d440afa7abf393c0ed4d63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ae4288789874729ab8826fe255b9220, a55f941fc2d440afa7abf393c0ed4d63; no voters:
I20260812 16:41:15.542179 5715 raft_consensus.cc:2781] T 74776322eae74e7aa7794747ab1a85c0 P a55f941fc2d440afa7abf393c0ed4d63 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:41:15.542390 5715 raft_consensus.cc:686] T 74776322eae74e7aa7794747ab1a85c0 P a55f941fc2d440afa7abf393c0ed4d63 [term 1 LEADER]: Becoming Leader. State: Replica: a55f941fc2d440afa7abf393c0ed4d63, State: Running, Role: LEADER
I20260812 16:41:15.542500 5715 consensus_queue.cc:237] T 74776322eae74e7aa7794747ab1a85c0 P a55f941fc2d440afa7abf393c0ed4d63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23adc9635529498aa929af64e7f1c232" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 34089 } } peers { permanent_uuid: "a55f941fc2d440afa7abf393c0ed4d63" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 34211 } } peers { permanent_uuid: "5ae4288789874729ab8826fe255b9220" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42791 } }
I20260812 16:41:15.543453 4851 catalog_manager.cc:5495] T 74776322eae74e7aa7794747ab1a85c0 P a55f941fc2d440afa7abf393c0ed4d63 reported cstate change: term changed from 0 to 1, leader changed from <none> to a55f941fc2d440afa7abf393c0ed4d63 (127.0.253.195). New cstate: current_term: 1 leader_uuid: "a55f941fc2d440afa7abf393c0ed4d63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23adc9635529498aa929af64e7f1c232" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 34089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a55f941fc2d440afa7abf393c0ed4d63" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 34211 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ae4288789874729ab8826fe255b9220" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42791 } health_report { overall_health: UNKNOWN } } }
I20260812 16:41:15.588197 5699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:15.585778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55004 (local address 127.0.253.195:34211)
0812 16:41:15.585850 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:15.585852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:15.585861 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:41:15.585911 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:15.585911 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:15.585949 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:15.586017 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:41:15.586159 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:15.586621 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:15.587459 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:15.587657 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:15.587713 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:15.587872 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:41:15.587978 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:41:15.588039 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:15.588080 (+ 41us) server_negotiation.cc:299] Negotiation successful
0812 16:41:15.588102 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 16:41:15.590301 5104 raft_consensus.cc:1261] T 74776322eae74e7aa7794747ab1a85c0 P 23adc9635529498aa929af64e7f1c232 [term 1 FOLLOWER]: Refusing update from remote peer a55f941fc2d440afa7abf393c0ed4d63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:41:15.590317 5346 raft_consensus.cc:1261] T 74776322eae74e7aa7794747ab1a85c0 P 5ae4288789874729ab8826fe255b9220 [term 1 FOLLOWER]: Refusing update from remote peer a55f941fc2d440afa7abf393c0ed4d63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:41:15.590747 5715 consensus_queue.cc:1035] T 74776322eae74e7aa7794747ab1a85c0 P a55f941fc2d440afa7abf393c0ed4d63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23adc9635529498aa929af64e7f1c232" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 34089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:41:15.590906 5720 consensus_queue.cc:1035] T 74776322eae74e7aa7794747ab1a85c0 P a55f941fc2d440afa7abf393c0ed4d63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ae4288789874729ab8826fe255b9220" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 42791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:41:15.593465 5730 mvcc.cc:205] Tried to move back new op lower bound from 7317720578414473216 to 7317720578222714880. Current Snapshot: MvccSnapshot[applied={T|T < 7317720578414473216}]
I20260812 16:41:15.593827 5734 mvcc.cc:205] Tried to move back new op lower bound from 7317720578414473216 to 7317720578222714880. Current Snapshot: MvccSnapshot[applied={T|T < 7317720578414473216}]
I20260812 16:41:15.596570 4851 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:41:15.596657 4851 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:41:15.597450 4851 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d715bf70f4c148ff8f5c85181bc5351b: Sending DeleteTablet for 3 replicas of tablet 74776322eae74e7aa7794747ab1a85c0
I20260812 16:41:15.597841 5545 tablet_service.cc:1514] Processing DeleteTablet for tablet 74776322eae74e7aa7794747ab1a85c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:41:15 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:47834
I20260812 16:41:15.597885 5312 tablet_service.cc:1514] Processing DeleteTablet for tablet 74776322eae74e7aa7794747ab1a85c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:41:15 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:58256
I20260812 16:41:15.598116 5748 tablet_replica.cc:330] T 74776322eae74e7aa7794747ab1a85c0 P a55f941fc2d440afa7abf393c0ed4d63: stopping tablet replica
I20260812 16:41:15.598131 5070 tablet_service.cc:1514] Processing DeleteTablet for tablet 74776322eae74e7aa7794747ab1a85c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:41:15 UTC) from {username='kudu', principal='kudu/127.0.253.254@KRBTEST.COM'} at 127.0.0.1:42460
I20260812 16:41:15.598240 5748 raft_consensus.cc:2229] T 74776322eae74e7aa7794747ab1a85c0 P a55f941fc2d440afa7abf393c0ed4d63 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:41:15.598376 5748 raft_consensus.cc:2258] T 74776322eae74e7aa7794747ab1a85c0 P a55f941fc2d440afa7abf393c0ed4d63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:41:15.598388 5750 tablet_replica.cc:330] T 74776322eae74e7aa7794747ab1a85c0 P 23adc9635529498aa929af64e7f1c232: stopping tablet replica
I20260812 16:41:15.598654 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 4962
I20260812 16:41:15.598829 5748 ts_tablet_manager.cc:1905] T 74776322eae74e7aa7794747ab1a85c0 P a55f941fc2d440afa7abf393c0ed4d63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:41:15.602203 5748 ts_tablet_manager.cc:1918] T 74776322eae74e7aa7794747ab1a85c0 P a55f941fc2d440afa7abf393c0ed4d63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:41:15.602290 5748 log.cc:1196] T 74776322eae74e7aa7794747ab1a85c0 P a55f941fc2d440afa7abf393c0ed4d63: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552619305671-1015-0/minicluster-data/ts-2/wal/wals/74776322eae74e7aa7794747ab1a85c0
I20260812 16:41:15.602591 5748 ts_tablet_manager.cc:1939] T 74776322eae74e7aa7794747ab1a85c0 P a55f941fc2d440afa7abf393c0ed4d63: Deleting consensus metadata
I20260812 16:41:15.603358 4819 catalog_manager.cc:4841] TS a55f941fc2d440afa7abf393c0ed4d63 (127.0.253.195:34211): tablet 74776322eae74e7aa7794747ab1a85c0 (table test-table [id=607bce51c8ed4ed4a475051cdcec2a59]) successfully deleted
I20260812 16:41:15.613047 5749 tablet_replica.cc:330] T 74776322eae74e7aa7794747ab1a85c0 P 5ae4288789874729ab8826fe255b9220: stopping tablet replica
W20260812 16:41:15.613106 4819 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.253.193:34089 (error 108)
I20260812 16:41:15.613225 5749 raft_consensus.cc:2229] T 74776322eae74e7aa7794747ab1a85c0 P 5ae4288789874729ab8826fe255b9220 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:41:15.613371 5749 raft_consensus.cc:2258] T 74776322eae74e7aa7794747ab1a85c0 P 5ae4288789874729ab8826fe255b9220 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:41:15.613811 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:15.613667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:34089 (local address 127.0.0.1:43112)
0812 16:41:15.613725 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:15.613769 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.193:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 16:41:15.613909 5749 ts_tablet_manager.cc:1905] T 74776322eae74e7aa7794747ab1a85c0 P 5ae4288789874729ab8826fe255b9220: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 16:41:15.613924 4819 catalog_manager.cc:4584] TS 23adc9635529498aa929af64e7f1c232 (127.0.253.193:34089): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 74776322eae74e7aa7794747ab1a85c0: Network error: Client connection negotiation failed: client connection to 127.0.253.193:34089: connect: Connection refused (error 111)
I20260812 16:41:15.614246 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 5213
I20260812 16:41:15.642668 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:15.642502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:42791 (local address 127.0.0.1:53834)
0812 16:41:15.642575 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:15.642611 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.194:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 16:41:15.642750 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 5427
I20260812 16:41:15.654278 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 4790
2026-08-12T16:41:15Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2498 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-12T16:41:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:41:15Z Disabled control of system clock
I20260812 16:41:15.694618 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:42597
--webserver_interface=127.0.253.254
--webserver_port=0
--builtin_ntp_servers=127.0.253.212:45399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:42597
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 16:41:15.781944 5759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:15.782171 5759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:15.782193 5759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:15.783296 5759 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:41:15.783331 5759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:15.783344 5759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:15.783356 5759 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:41:15.783368 5759 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:41:15.784657 5759 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:45399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:42597
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:42597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:15.784857 5759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:15.785095 5759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:15.790338 5767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:15.790333 5766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:15.790405 5771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:15.790537 5759 server_base.cc:1034] running on GCE node
I20260812 16:41:15.790915 5759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:15.791271 5759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:15.792448 5759 hybrid_clock.cc:648] HybridClock initialized: now 1786552875792422 us; error 47 us; skew 500 ppm
I20260812 16:41:15.793879 5759 webserver.cc:460] Webserver started at http://127.0.253.254:41887/ using document root <none> and password file <none>
I20260812 16:41:15.794152 5759 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:15.794229 5759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:15.794384 5759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:15.795590 5759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "0314014c83964e6092d1826632d2b3bc"
format_stamp: "Formatted at 2026-08-12 16:41:15 on dist-test-slave-w4v5"
server_key: "40d85a36f53ef3afc34ba159e0d54684"
server_key_iv: "d03c4e810c459edaf7c6bbab952da4e0"
server_key_version: "encryptionkey@0"
I20260812 16:41:15.795971 5759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "0314014c83964e6092d1826632d2b3bc"
format_stamp: "Formatted at 2026-08-12 16:41:15 on dist-test-slave-w4v5"
server_key: "40d85a36f53ef3afc34ba159e0d54684"
server_key_iv: "d03c4e810c459edaf7c6bbab952da4e0"
server_key_version: "encryptionkey@0"
I20260812 16:41:15.797396 5759 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:41:15.798199 5785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:15.798398 5759 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:41:15.798509 5759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "0314014c83964e6092d1826632d2b3bc"
format_stamp: "Formatted at 2026-08-12 16:41:15 on dist-test-slave-w4v5"
server_key: "40d85a36f53ef3afc34ba159e0d54684"
server_key_iv: "d03c4e810c459edaf7c6bbab952da4e0"
server_key_version: "encryptionkey@0"
I20260812 16:41:15.798591 5759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:15.821987 5759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:15.822361 5759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:15.822533 5759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:15.826998 5759 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:42597
I20260812 16:41:15.827042 5882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:42597 every 8 connection(s)
I20260812 16:41:15.827476 5759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:41:15.828567 5885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:15.829913 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 5759
I20260812 16:41:15.830052 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:41:15.830256 1015 external_mini_cluster.cc:1428] Setting key 6af2701cdf14d985e9618b73caff6cae
I20260812 16:41:15.831526 5885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0314014c83964e6092d1826632d2b3bc: Bootstrap starting.
I20260812 16:41:15.832304 5885 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0314014c83964e6092d1826632d2b3bc: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:15.832616 5885 log.cc:824] T 00000000000000000000000000000000 P 0314014c83964e6092d1826632d2b3bc: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:15.833367 5885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0314014c83964e6092d1826632d2b3bc: No bootstrap required, opened a new log
I20260812 16:41:15.835274 5885 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0314014c83964e6092d1826632d2b3bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0314014c83964e6092d1826632d2b3bc" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 42597 } }
I20260812 16:41:15.835433 5885 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0314014c83964e6092d1826632d2b3bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:15.835484 5885 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0314014c83964e6092d1826632d2b3bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0314014c83964e6092d1826632d2b3bc, State: Initialized, Role: FOLLOWER
I20260812 16:41:15.835651 5885 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0314014c83964e6092d1826632d2b3bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0314014c83964e6092d1826632d2b3bc" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 42597 } }
I20260812 16:41:15.835755 5885 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0314014c83964e6092d1826632d2b3bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:41:15.835791 5885 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0314014c83964e6092d1826632d2b3bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:41:15.835841 5885 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0314014c83964e6092d1826632d2b3bc [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:15.836103 5888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:15.830869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49560 (local address 127.0.253.254:42597)
0812 16:41:15.831125 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:15.831136 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:15.831181 (+ 45us) server_negotiation.cc:407] Connection header received
0812 16:41:15.831566 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:15.831572 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:15.831659 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:15.831794 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:15.832058 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:15.833019 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:15.833920 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:15.834342 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:15.834482 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:15.835249 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:15.835254 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:15.835254 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:41:15.835432 (+ 178us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:15.835491 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:41:15.835508 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:15.835602 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:15.835722 (+ 120us) server_negotiation.cc:299] Negotiation successful
0812 16:41:15.835928 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":93,"threads_started":1}
I20260812 16:41:15.836568 5885 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0314014c83964e6092d1826632d2b3bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0314014c83964e6092d1826632d2b3bc" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 42597 } }
I20260812 16:41:15.836676 5885 leader_election.cc:302] T 00000000000000000000000000000000 P 0314014c83964e6092d1826632d2b3bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0314014c83964e6092d1826632d2b3bc; no voters:
I20260812 16:41:15.836851 5885 leader_election.cc:288] T 00000000000000000000000000000000 P 0314014c83964e6092d1826632d2b3bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:41:15.836874 5890 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0314014c83964e6092d1826632d2b3bc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:41:15.837004 5890 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0314014c83964e6092d1826632d2b3bc [term 1 LEADER]: Becoming Leader. State: Replica: 0314014c83964e6092d1826632d2b3bc, State: Running, Role: LEADER
I20260812 16:41:15.837095 5890 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0314014c83964e6092d1826632d2b3bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0314014c83964e6092d1826632d2b3bc" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 42597 } }
I20260812 16:41:15.837154 5885 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0314014c83964e6092d1826632d2b3bc [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:41:15.837422 5891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0314014c83964e6092d1826632d2b3bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0314014c83964e6092d1826632d2b3bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0314014c83964e6092d1826632d2b3bc" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 42597 } } }
I20260812 16:41:15.837422 5893 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0314014c83964e6092d1826632d2b3bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0314014c83964e6092d1826632d2b3bc. Latest consensus state: current_term: 1 leader_uuid: "0314014c83964e6092d1826632d2b3bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0314014c83964e6092d1826632d2b3bc" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 42597 } } }
I20260812 16:41:15.837630 5893 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0314014c83964e6092d1826632d2b3bc [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:15.837846 5891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0314014c83964e6092d1826632d2b3bc [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:15.838231 5900 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:41:15.838795 5900 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:41:15.840528 5900 catalog_manager.cc:1347] Generated new cluster ID: fb1422939a404e019f48efe4e80dc2c3
I20260812 16:41:15.840610 5900 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:41:15.861104 5900 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:41:15.861735 5900 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:41:15.865447 5900 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0314014c83964e6092d1826632d2b3bc: Generated new TSK 0
I20260812 16:41:15.865672 5900 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:41:15.870178 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.193:0
--local_ip_for_outbound_sockets=127.0.253.193
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:42597
--builtin_ntp_servers=127.0.253.212:45399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 16:41:15.957361 5931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:15.957607 5931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:15.957643 5931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:15.958791 5931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:15.958860 5931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:15.958900 5931 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.193
I20260812 16:41:15.960346 5931 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:45399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.253.193
--webserver_port=0
--tserver_master_addrs=127.0.253.254:42597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.193
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:15.960630 5931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:15.960958 5931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:15.966174 5943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:15.966177 5939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:15.966177 5940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:15.966759 5931 server_base.cc:1034] running on GCE node
I20260812 16:41:15.966991 5931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:15.967272 5931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:15.968466 5931 hybrid_clock.cc:648] HybridClock initialized: now 1786552875968449 us; error 31 us; skew 500 ppm
I20260812 16:41:15.970109 5931 webserver.cc:460] Webserver started at http://127.0.253.193:44485/ using document root <none> and password file <none>
I20260812 16:41:15.970397 5931 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:15.970477 5931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:15.970669 5931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:15.971983 5931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-0/data/instance:
uuid: "40ad1847223a42139c4cb6ced039231c"
format_stamp: "Formatted at 2026-08-12 16:41:15 on dist-test-slave-w4v5"
server_key: "be18f8fafc3cd14388e9441d32ae6b96"
server_key_iv: "faa7e18c753c68e3972884c72b8d809c"
server_key_version: "encryptionkey@0"
I20260812 16:41:15.972410 5931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance:
uuid: "40ad1847223a42139c4cb6ced039231c"
format_stamp: "Formatted at 2026-08-12 16:41:15 on dist-test-slave-w4v5"
server_key: "be18f8fafc3cd14388e9441d32ae6b96"
server_key_iv: "faa7e18c753c68e3972884c72b8d809c"
server_key_version: "encryptionkey@0"
I20260812 16:41:15.974031 5931 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260812 16:41:15.974798 5953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:15.974975 5931 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:41:15.975128 5931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-0/wal
uuid: "40ad1847223a42139c4cb6ced039231c"
format_stamp: "Formatted at 2026-08-12 16:41:15 on dist-test-slave-w4v5"
server_key: "be18f8fafc3cd14388e9441d32ae6b96"
server_key_iv: "faa7e18c753c68e3972884c72b8d809c"
server_key_version: "encryptionkey@0"
I20260812 16:41:15.975217 5931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:15.992467 5931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:15.992858 5931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:15.993032 5931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:15.993304 5931 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:41:15.993692 5931 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:41:15.993757 5931 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:15.993804 5931 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:41:15.993840 5931 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:16.000474 5931 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.193:41699
I20260812 16:41:16.000517 6162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.193:41699 every 8 connection(s)
I20260812 16:41:16.000926 5931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-0/data/info.pb
I20260812 16:41:16.006781 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 5931
I20260812 16:41:16.006811 5888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:16.001838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.193:43903 (local address 127.0.253.254:42597)
0812 16:41:16.001913 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:16.001916 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:16.002233 (+ 317us) server_negotiation.cc:407] Connection header received
0812 16:41:16.002874 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:16.002875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:16.002886 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:16.002957 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:16.003783 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:16.004334 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:16.005349 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:16.005560 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:16.005669 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:16.006481 (+ 812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:16.006482 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:16.006483 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:41:16.006487 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:16.006501 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:41:16.006505 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:16.006610 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:16.006673 (+ 63us) server_negotiation.cc:299] Negotiation successful
0812 16:41:16.006706 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 16:41:16.006901 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-0/wal/instance
I20260812 16:41:16.006964 6165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:16.001789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:42597 (local address 127.0.253.193:43903)
0812 16:41:16.002015 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:16.002057 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:16.002608 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:16.003047 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:16.003050 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:16.003059 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:41:16.003689 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:16.003694 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:16.004430 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:16.004431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:16.005255 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:16.005257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:16.005685 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:16.005686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:16.005802 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:16.006344 (+ 542us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:41:16.006345 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:41:16.006375 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:41:16.006398 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:41:16.006611 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:41:16.006615 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:41:16.006618 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:41:16.006664 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 16:41:16.006747 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":105,"threads_started":1}
I20260812 16:41:16.007220 1015 external_mini_cluster.cc:1428] Setting key 9432d2d0d616fb69a2c36e37188441bc
I20260812 16:41:16.007552 6163 heartbeater.cc:347] Connected to a master server at 127.0.253.254:42597
I20260812 16:41:16.007669 6163 heartbeater.cc:464] Registering TS with master...
I20260812 16:41:16.007910 6163 heartbeater.cc:511] Master 127.0.253.254:42597 requested a full tablet report, sending...
I20260812 16:41:16.008349 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.194:0
--local_ip_for_outbound_sockets=127.0.253.194
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:42597
--builtin_ntp_servers=127.0.253.212:45399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 16:41:16.008580 5818 ts_manager.cc:194] Registered new tserver with Master: 40ad1847223a42139c4cb6ced039231c (127.0.253.193:41699)
I20260812 16:41:16.009241 5818 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.253.193:43903
W20260812 16:41:16.094233 6166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:16.094471 6166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:16.094506 6166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:16.095651 6166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:16.095715 6166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:16.095757 6166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.194
I20260812 16:41:16.097114 6166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:45399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.253.194
--webserver_port=0
--tserver_master_addrs=127.0.253.254:42597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.194
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:16.097388 6166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:16.097702 6166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:16.102869 6175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:16.102945 6173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:16.102957 6178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:16.103473 6166 server_base.cc:1034] running on GCE node
I20260812 16:41:16.103675 6166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:16.103873 6166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:16.105062 6166 hybrid_clock.cc:648] HybridClock initialized: now 1786552876105028 us; error 39 us; skew 500 ppm
I20260812 16:41:16.106411 6166 webserver.cc:460] Webserver started at http://127.0.253.194:44059/ using document root <none> and password file <none>
I20260812 16:41:16.106627 6166 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:16.106669 6166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:16.106762 6166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:16.107847 6166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-1/data/instance:
uuid: "6cf3f09ccf8b471298f86d9125a78c48"
format_stamp: "Formatted at 2026-08-12 16:41:16 on dist-test-slave-w4v5"
server_key: "4810d1b3db78daac544d2fd55d764d77"
server_key_iv: "fbe81ae8ceb2e355b4e196aee550603d"
server_key_version: "encryptionkey@0"
I20260812 16:41:16.108178 6166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance:
uuid: "6cf3f09ccf8b471298f86d9125a78c48"
format_stamp: "Formatted at 2026-08-12 16:41:16 on dist-test-slave-w4v5"
server_key: "4810d1b3db78daac544d2fd55d764d77"
server_key_iv: "fbe81ae8ceb2e355b4e196aee550603d"
server_key_version: "encryptionkey@0"
I20260812 16:41:16.109479 6166 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:41:16.110255 6186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:16.110460 6166 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:41:16.110605 6166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-1/wal
uuid: "6cf3f09ccf8b471298f86d9125a78c48"
format_stamp: "Formatted at 2026-08-12 16:41:16 on dist-test-slave-w4v5"
server_key: "4810d1b3db78daac544d2fd55d764d77"
server_key_iv: "fbe81ae8ceb2e355b4e196aee550603d"
server_key_version: "encryptionkey@0"
I20260812 16:41:16.110678 6166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:16.122671 6166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:16.122967 6166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:16.123169 6166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:16.123394 6166 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:41:16.123764 6166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:41:16.123821 6166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:16.123863 6166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:41:16.123891 6166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:16.129666 6166 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.194:37333
I20260812 16:41:16.129722 6380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.194:37333 every 8 connection(s)
I20260812 16:41:16.130039 6166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-1/data/info.pb
I20260812 16:41:16.134541 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 6166
I20260812 16:41:16.134655 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-1/wal/instance
I20260812 16:41:16.134873 1015 external_mini_cluster.cc:1428] Setting key 623afb99f152f0867e6705ff775c675d
I20260812 16:41:16.135630 5888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:16.130779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.194:50583 (local address 127.0.253.254:42597)
0812 16:41:16.130827 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:16.130829 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:16.131036 (+ 207us) server_negotiation.cc:407] Connection header received
0812 16:41:16.131629 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:16.131630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:16.131643 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:16.131717 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:16.132406 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:16.132908 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:16.133919 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:16.134123 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:16.134200 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:16.135347 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:16.135348 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:16.135349 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:41:16.135353 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:16.135370 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:41:16.135374 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:16.135431 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:16.135549 (+ 118us) server_negotiation.cc:299] Negotiation successful
0812 16:41:16.135571 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 16:41:16.135943 6384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:16.130737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:42597 (local address 127.0.253.194:50583)
0812 16:41:16.130906 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:16.130945 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:16.131516 (+ 571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:16.131787 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:16.131792 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:16.131802 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:41:16.132310 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:16.132315 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:16.133010 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:16.133012 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:16.133820 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:16.133823 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:16.134220 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:16.134222 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:16.134400 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:16.135181 (+ 781us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:41:16.135182 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:41:16.135225 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:41:16.135258 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:41:16.135455 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:41:16.135462 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:41:16.135466 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:41:16.135542 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 16:41:16.135664 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":64,"threads_started":1}
I20260812 16:41:16.136132 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.253.195:0
--local_ip_for_outbound_sockets=127.0.253.195
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:42597
--builtin_ntp_servers=127.0.253.212:45399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 16:41:16.136587 6381 heartbeater.cc:347] Connected to a master server at 127.0.253.254:42597
I20260812 16:41:16.136699 6381 heartbeater.cc:464] Registering TS with master...
I20260812 16:41:16.136906 6381 heartbeater.cc:511] Master 127.0.253.254:42597 requested a full tablet report, sending...
I20260812 16:41:16.137295 5818 ts_manager.cc:194] Registered new tserver with Master: 6cf3f09ccf8b471298f86d9125a78c48 (127.0.253.194:37333)
I20260812 16:41:16.137815 5818 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.253.194:50583
W20260812 16:41:16.221020 6385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:16.221264 6385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:16.221303 6385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:16.222450 6385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:16.222572 6385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:16.222621 6385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.253.195
I20260812 16:41:16.224093 6385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:45399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.253.195
--webserver_port=0
--tserver_master_addrs=127.0.253.254:42597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.253.195
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:16.224365 6385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:16.224677 6385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:16.229876 6397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:16.229897 6393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:16.229854 6394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:16.230103 6385 server_base.cc:1034] running on GCE node
I20260812 16:41:16.230330 6385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:16.230597 6385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:16.231781 6385 hybrid_clock.cc:648] HybridClock initialized: now 1786552876231761 us; error 53 us; skew 500 ppm
I20260812 16:41:16.233100 6385 webserver.cc:460] Webserver started at http://127.0.253.195:33077/ using document root <none> and password file <none>
I20260812 16:41:16.233379 6385 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:16.233453 6385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:16.233649 6385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:16.234731 6385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-2/data/instance:
uuid: "e3ebcb73a5654f9590655ca5a454a013"
format_stamp: "Formatted at 2026-08-12 16:41:16 on dist-test-slave-w4v5"
server_key: "c50f28395c2dd31b6f23a02f801ccbea"
server_key_iv: "30c2b4c3012e2e1909fc0d60ef2390d6"
server_key_version: "encryptionkey@0"
I20260812 16:41:16.235198 6385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance:
uuid: "e3ebcb73a5654f9590655ca5a454a013"
format_stamp: "Formatted at 2026-08-12 16:41:16 on dist-test-slave-w4v5"
server_key: "c50f28395c2dd31b6f23a02f801ccbea"
server_key_iv: "30c2b4c3012e2e1909fc0d60ef2390d6"
server_key_version: "encryptionkey@0"
I20260812 16:41:16.236649 6385 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:41:16.237521 6405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:16.237735 6385 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:41:16.237831 6385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-2/wal
uuid: "e3ebcb73a5654f9590655ca5a454a013"
format_stamp: "Formatted at 2026-08-12 16:41:16 on dist-test-slave-w4v5"
server_key: "c50f28395c2dd31b6f23a02f801ccbea"
server_key_iv: "30c2b4c3012e2e1909fc0d60ef2390d6"
server_key_version: "encryptionkey@0"
I20260812 16:41:16.237914 6385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:16.248953 6385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:16.249281 6385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:16.249441 6385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:16.249687 6385 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:41:16.250046 6385 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:41:16.250103 6385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:16.250139 6385 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:41:16.250166 6385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:16.256171 6385 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.195:32809
I20260812 16:41:16.256278 6603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.195:32809 every 8 connection(s)
I20260812 16:41:16.256589 6385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-2/data/info.pb
I20260812 16:41:16.262112 5888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:16.257330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:48369 (local address 127.0.253.254:42597)
0812 16:41:16.257402 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:16.257404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:16.257708 (+ 304us) server_negotiation.cc:407] Connection header received
0812 16:41:16.258239 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:16.258240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:16.258254 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:16.258319 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:16.259032 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:16.259570 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:16.260594 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:16.260818 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:16.260899 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:16.261799 (+ 900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:16.261800 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:16.261801 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:41:16.261805 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:16.261821 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:41:16.261825 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:16.261869 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:16.261977 (+ 108us) server_negotiation.cc:299] Negotiation successful
0812 16:41:16.262005 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 16:41:16.262310 6606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:16.257303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:42597 (local address 127.0.253.195:48369)
0812 16:41:16.257539 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:16.257584 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:16.258118 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:16.258393 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:16.258397 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:16.258407 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:41:16.258933 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:16.258938 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:16.259695 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:16.259696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:16.260498 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:16.260500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:16.260907 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:16.260909 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:16.261051 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:16.261642 (+ 591us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:41:16.261643 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:41:16.261673 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:41:16.261696 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:41:16.261880 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:41:16.261884 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:41:16.261887 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:41:16.261967 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 16:41:16.262071 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":90,"threads_started":1}
I20260812 16:41:16.262379 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 6385
I20260812 16:41:16.262494 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-2/wal/instance
I20260812 16:41:16.262790 1015 external_mini_cluster.cc:1428] Setting key ef2502137607f93145098a05aa36e1c0
I20260812 16:41:16.262792 6604 heartbeater.cc:347] Connected to a master server at 127.0.253.254:42597
I20260812 16:41:16.262887 6604 heartbeater.cc:464] Registering TS with master...
I20260812 16:41:16.263132 6604 heartbeater.cc:511] Master 127.0.253.254:42597 requested a full tablet report, sending...
I20260812 16:41:16.263545 5818 ts_manager.cc:194] Registered new tserver with Master: e3ebcb73a5654f9590655ca5a454a013 (127.0.253.195:32809)
I20260812 16:41:16.263854 1015 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:41:16.264043 5818 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.253.195:48369
I20260812 16:41:16.268363 5888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:16.265535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49572 (local address 127.0.253.254:42597)
0812 16:41:16.265608 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:16.265610 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:16.265660 (+ 50us) server_negotiation.cc:407] Connection header received
0812 16:41:16.265707 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:16.265707 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:16.265715 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:16.265767 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:16.265881 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:16.266324 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:16.267127 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:16.267364 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:16.267432 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:16.268110 (+ 678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:16.268111 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:16.268112 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:41:16.268116 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:16.268129 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:41:16.268134 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:16.268187 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:16.268233 (+ 46us) server_negotiation.cc:299] Negotiation successful
0812 16:41:16.268254 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 16:41:16.270537 5818 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49572:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:41:16.272197 5818 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:41:16.279323 6623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:16.274875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59656 (local address 127.0.253.193:41699)
0812 16:41:16.275391 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:16.275394 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:16.275403 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:41:16.275436 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:16.275456 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:16.275472 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:16.275549 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:41:16.275781 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:16.276917 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:16.278060 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:16.278856 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:16.278928 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:16.279029 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:16.279188 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 16:41:16.279232 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":90,"threads_started":1}
I20260812 16:41:16.279419 6622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:16.274759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49644 (local address 127.0.253.194:37333)
0812 16:41:16.274936 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:16.274941 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:16.275303 (+ 362us) server_negotiation.cc:407] Connection header received
0812 16:41:16.275357 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:16.275359 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:16.275396 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:16.275470 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:41:16.275769 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:16.276921 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:16.278338 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:16.278923 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:16.278994 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:16.279129 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:16.279278 (+ 149us) server_negotiation.cc:299] Negotiation successful
0812 16:41:16.279324 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":108,"threads_started":1}
I20260812 16:41:16.279644 6624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:16.274668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:41699 (local address 127.0.0.1:59656)
0812 16:41:16.275046 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:16.275129 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:16.275209 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:16.275616 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:16.275616 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:16.275618 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:41:16.275681 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:16.275683 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:16.277058 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:16.277059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:16.277960 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:16.277963 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:16.278947 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:16.278949 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:16.279027 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:16.279044 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 16:41:16.279461 (+ 417us) client_negotiation.cc:240] Negotiation successful
0812 16:41:16.279564 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"mutex_wait_us":16,"thread_start_us":84,"threads_started":1}
I20260812 16:41:16.279891 6621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:16.274668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:37333 (local address 127.0.0.1:49644)
0812 16:41:16.275092 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:16.275128 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:16.275209 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:16.275601 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:16.275604 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:16.275606 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:41:16.275683 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:16.275684 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:16.277006 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:16.277006 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:16.278229 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:16.278232 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:16.279131 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:16.279132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:16.279195 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:16.279199 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:41:16.279262 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 16:41:16.279340 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"mutex_wait_us":43,"spinlock_wait_cycles":384,"thread_start_us":93,"threads_started":1}
I20260812 16:41:16.280078 6625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:16.275162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39842 (local address 127.0.253.195:32809)
0812 16:41:16.275679 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:16.275684 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:16.275693 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:41:16.275729 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:16.275731 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:16.275746 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:16.275817 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:41:16.276004 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:16.276727 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:16.278360 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:16.279541 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:16.279611 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:16.279690 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:16.279944 (+ 254us) server_negotiation.cc:299] Negotiation successful
0812 16:41:16.279989 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":101,"threads_started":1}
I20260812 16:41:16.279902 6620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:16.274893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.195:32809 (local address 127.0.0.1:39842)
0812 16:41:16.275158 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:16.275168 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:16.275211 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:16.275885 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:16.275886 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:16.275888 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:41:16.275923 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:16.275924 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:16.276905 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:16.276906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:16.278265 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:16.278267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:16.279624 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:16.279625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:16.279689 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:16.279693 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:41:16.279753 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 16:41:16.279796 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":30,"thread_start_us":77,"threads_started":1}
I20260812 16:41:16.280392 6031 tablet_service.cc:1467] Processing CreateTablet for tablet 030f6aebc2f04b648fbd6ba3cdf4bd5f (DEFAULT_TABLE table=test-table [id=1acffdd9279f4689b5401cdd505fff79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:41:16.280797 6031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 030f6aebc2f04b648fbd6ba3cdf4bd5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:16.281075 6265 tablet_service.cc:1467] Processing CreateTablet for tablet 030f6aebc2f04b648fbd6ba3cdf4bd5f (DEFAULT_TABLE table=test-table [id=1acffdd9279f4689b5401cdd505fff79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:41:16.281450 6265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 030f6aebc2f04b648fbd6ba3cdf4bd5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:16.283782 6628 tablet_bootstrap.cc:492] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P 40ad1847223a42139c4cb6ced039231c: Bootstrap starting.
I20260812 16:41:16.283913 6488 tablet_service.cc:1467] Processing CreateTablet for tablet 030f6aebc2f04b648fbd6ba3cdf4bd5f (DEFAULT_TABLE table=test-table [id=1acffdd9279f4689b5401cdd505fff79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:41:16.284301 6488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 030f6aebc2f04b648fbd6ba3cdf4bd5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:16.284341 6631 tablet_bootstrap.cc:492] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P 6cf3f09ccf8b471298f86d9125a78c48: Bootstrap starting.
I20260812 16:41:16.284587 6628 tablet_bootstrap.cc:654] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P 40ad1847223a42139c4cb6ced039231c: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:16.284930 6628 log.cc:824] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P 40ad1847223a42139c4cb6ced039231c: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:16.285097 6631 tablet_bootstrap.cc:654] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P 6cf3f09ccf8b471298f86d9125a78c48: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:16.285380 6631 log.cc:824] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P 6cf3f09ccf8b471298f86d9125a78c48: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:16.286149 6631 tablet_bootstrap.cc:492] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P 6cf3f09ccf8b471298f86d9125a78c48: No bootstrap required, opened a new log
I20260812 16:41:16.286157 6628 tablet_bootstrap.cc:492] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P 40ad1847223a42139c4cb6ced039231c: No bootstrap required, opened a new log
I20260812 16:41:16.286219 6631 ts_tablet_manager.cc:1397] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P 6cf3f09ccf8b471298f86d9125a78c48: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:41:16.286219 6628 ts_tablet_manager.cc:1397] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P 40ad1847223a42139c4cb6ced039231c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:41:16.287619 6636 tablet_bootstrap.cc:492] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P e3ebcb73a5654f9590655ca5a454a013: Bootstrap starting.
I20260812 16:41:16.288239 6636 tablet_bootstrap.cc:654] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P e3ebcb73a5654f9590655ca5a454a013: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:16.288558 6636 log.cc:824] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P e3ebcb73a5654f9590655ca5a454a013: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:16.288902 6631 raft_consensus.cc:348] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P 6cf3f09ccf8b471298f86d9125a78c48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40ad1847223a42139c4cb6ced039231c" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 41699 } } peers { permanent_uuid: "e3ebcb73a5654f9590655ca5a454a013" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32809 } } peers { permanent_uuid: "6cf3f09ccf8b471298f86d9125a78c48" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37333 } }
I20260812 16:41:16.288902 6628 raft_consensus.cc:348] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P 40ad1847223a42139c4cb6ced039231c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40ad1847223a42139c4cb6ced039231c" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 41699 } } peers { permanent_uuid: "e3ebcb73a5654f9590655ca5a454a013" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32809 } } peers { permanent_uuid: "6cf3f09ccf8b471298f86d9125a78c48" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37333 } }
I20260812 16:41:16.289073 6631 raft_consensus.cc:374] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P 6cf3f09ccf8b471298f86d9125a78c48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:16.289073 6628 raft_consensus.cc:374] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P 40ad1847223a42139c4cb6ced039231c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:16.289117 6631 raft_consensus.cc:729] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P 6cf3f09ccf8b471298f86d9125a78c48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cf3f09ccf8b471298f86d9125a78c48, State: Initialized, Role: FOLLOWER
I20260812 16:41:16.289117 6628 raft_consensus.cc:729] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P 40ad1847223a42139c4cb6ced039231c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40ad1847223a42139c4cb6ced039231c, State: Initialized, Role: FOLLOWER
I20260812 16:41:16.289238 6628 consensus_queue.cc:260] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P 40ad1847223a42139c4cb6ced039231c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40ad1847223a42139c4cb6ced039231c" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 41699 } } peers { permanent_uuid: "e3ebcb73a5654f9590655ca5a454a013" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32809 } } peers { permanent_uuid: "6cf3f09ccf8b471298f86d9125a78c48" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37333 } }
I20260812 16:41:16.289275 6631 consensus_queue.cc:260] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P 6cf3f09ccf8b471298f86d9125a78c48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40ad1847223a42139c4cb6ced039231c" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 41699 } } peers { permanent_uuid: "e3ebcb73a5654f9590655ca5a454a013" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32809 } } peers { permanent_uuid: "6cf3f09ccf8b471298f86d9125a78c48" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37333 } }
I20260812 16:41:16.289343 6636 tablet_bootstrap.cc:492] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P e3ebcb73a5654f9590655ca5a454a013: No bootstrap required, opened a new log
I20260812 16:41:16.289412 6636 ts_tablet_manager.cc:1397] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P e3ebcb73a5654f9590655ca5a454a013: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:41:16.289516 6628 ts_tablet_manager.cc:1428] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P 40ad1847223a42139c4cb6ced039231c: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260812 16:41:16.289516 6631 ts_tablet_manager.cc:1428] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P 6cf3f09ccf8b471298f86d9125a78c48: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260812 16:41:16.289644 6381 heartbeater.cc:503] Master 127.0.253.254:42597 was elected leader, sending a full tablet report...
I20260812 16:41:16.289661 6163 heartbeater.cc:503] Master 127.0.253.254:42597 was elected leader, sending a full tablet report...
I20260812 16:41:16.291985 6636 raft_consensus.cc:348] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P e3ebcb73a5654f9590655ca5a454a013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40ad1847223a42139c4cb6ced039231c" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 41699 } } peers { permanent_uuid: "e3ebcb73a5654f9590655ca5a454a013" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32809 } } peers { permanent_uuid: "6cf3f09ccf8b471298f86d9125a78c48" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37333 } }
I20260812 16:41:16.292158 6636 raft_consensus.cc:374] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P e3ebcb73a5654f9590655ca5a454a013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:16.292207 6636 raft_consensus.cc:729] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P e3ebcb73a5654f9590655ca5a454a013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3ebcb73a5654f9590655ca5a454a013, State: Initialized, Role: FOLLOWER
I20260812 16:41:16.292323 6636 consensus_queue.cc:260] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P e3ebcb73a5654f9590655ca5a454a013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40ad1847223a42139c4cb6ced039231c" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 41699 } } peers { permanent_uuid: "e3ebcb73a5654f9590655ca5a454a013" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32809 } } peers { permanent_uuid: "6cf3f09ccf8b471298f86d9125a78c48" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37333 } }
I20260812 16:41:16.292605 6636 ts_tablet_manager.cc:1428] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P e3ebcb73a5654f9590655ca5a454a013: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:41:16.292721 6604 heartbeater.cc:503] Master 127.0.253.254:42597 was elected leader, sending a full tablet report...
W20260812 16:41:16.380595 6383 tablet.cc:2369] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P 6cf3f09ccf8b471298f86d9125a78c48: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:41:16.463279 6641 raft_consensus.cc:482] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P e3ebcb73a5654f9590655ca5a454a013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:41:16.463416 6641 raft_consensus.cc:504] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P e3ebcb73a5654f9590655ca5a454a013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40ad1847223a42139c4cb6ced039231c" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 41699 } } peers { permanent_uuid: "e3ebcb73a5654f9590655ca5a454a013" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32809 } } peers { permanent_uuid: "6cf3f09ccf8b471298f86d9125a78c48" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37333 } }
I20260812 16:41:16.463768 6641 leader_election.cc:288] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P e3ebcb73a5654f9590655ca5a454a013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 40ad1847223a42139c4cb6ced039231c (127.0.253.193:41699), 6cf3f09ccf8b471298f86d9125a78c48 (127.0.253.194:37333)
I20260812 16:41:16.466801 6606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:16.463993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:41699 (local address 127.0.253.195:50679)
0812 16:41:16.464046 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:16.464060 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:16.464126 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:16.464297 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:16.464298 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:16.464300 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:41:16.464385 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:16.464403 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:16.465125 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:16.465126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:16.465969 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:16.465971 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:16.466532 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:16.466533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:16.466604 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:16.466608 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:41:16.466673 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 16:41:16.466722 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 16:41:16.466801 6623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:16.464087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:50679 (local address 127.0.253.193:41699)
0812 16:41:16.464159 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:16.464167 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:16.464176 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:41:16.464202 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:16.464203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:16.464213 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:16.464309 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:41:16.464512 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:16.465023 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:16.466066 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:16.466432 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:16.466525 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:16.466535 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:16.466694 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 16:41:16.466737 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 16:41:16.467115 6622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:16.464090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.253.195:42467 (local address 127.0.253.194:37333)
0812 16:41:16.464162 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:16.464164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:16.464480 (+ 316us) server_negotiation.cc:407] Connection header received
0812 16:41:16.464544 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:16.464545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:16.464554 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:16.464637 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:41:16.464784 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:16.465246 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:16.466322 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:16.466699 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:16.466789 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:16.466797 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:16.466971 (+ 174us) server_negotiation.cc:299] Negotiation successful
0812 16:41:16.467024 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 16:41:16.467190 6082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "030f6aebc2f04b648fbd6ba3cdf4bd5f" candidate_uuid: "e3ebcb73a5654f9590655ca5a454a013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40ad1847223a42139c4cb6ced039231c" is_pre_election: true
I20260812 16:41:16.467339 6642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:16.463993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.194:37333 (local address 127.0.253.195:42467)
0812 16:41:16.464393 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:16.464423 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:16.464484 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:16.464633 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:16.464634 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:16.464635 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:41:16.464712 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:16.464714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:16.465341 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:16.465342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:16.466227 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:16.466230 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:16.466799 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:16.466800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:16.466873 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:16.466877 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:41:16.467215 (+ 338us) client_negotiation.cc:240] Negotiation successful
0812 16:41:16.467260 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"mutex_wait_us":177,"thread_start_us":118,"threads_started":1}
I20260812 16:41:16.467347 6082 raft_consensus.cc:2454] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P 40ad1847223a42139c4cb6ced039231c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3ebcb73a5654f9590655ca5a454a013 in term 0.
I20260812 16:41:16.467578 6411 leader_election.cc:302] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P e3ebcb73a5654f9590655ca5a454a013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40ad1847223a42139c4cb6ced039231c, e3ebcb73a5654f9590655ca5a454a013; no voters:
I20260812 16:41:16.467736 6641 raft_consensus.cc:2781] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P e3ebcb73a5654f9590655ca5a454a013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:41:16.467722 6304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "030f6aebc2f04b648fbd6ba3cdf4bd5f" candidate_uuid: "e3ebcb73a5654f9590655ca5a454a013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cf3f09ccf8b471298f86d9125a78c48" is_pre_election: true
I20260812 16:41:16.467808 6641 raft_consensus.cc:482] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P e3ebcb73a5654f9590655ca5a454a013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:41:16.467835 6641 raft_consensus.cc:3037] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P e3ebcb73a5654f9590655ca5a454a013 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:16.467895 6304 raft_consensus.cc:2454] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P 6cf3f09ccf8b471298f86d9125a78c48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3ebcb73a5654f9590655ca5a454a013 in term 0.
I20260812 16:41:16.468655 6641 raft_consensus.cc:504] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P e3ebcb73a5654f9590655ca5a454a013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40ad1847223a42139c4cb6ced039231c" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 41699 } } peers { permanent_uuid: "e3ebcb73a5654f9590655ca5a454a013" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32809 } } peers { permanent_uuid: "6cf3f09ccf8b471298f86d9125a78c48" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37333 } }
I20260812 16:41:16.468824 6641 leader_election.cc:288] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P e3ebcb73a5654f9590655ca5a454a013 [CANDIDATE]: Term 1 election: Requested vote from peers 40ad1847223a42139c4cb6ced039231c (127.0.253.193:41699), 6cf3f09ccf8b471298f86d9125a78c48 (127.0.253.194:37333)
I20260812 16:41:16.468964 6082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "030f6aebc2f04b648fbd6ba3cdf4bd5f" candidate_uuid: "e3ebcb73a5654f9590655ca5a454a013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40ad1847223a42139c4cb6ced039231c"
I20260812 16:41:16.468966 6304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "030f6aebc2f04b648fbd6ba3cdf4bd5f" candidate_uuid: "e3ebcb73a5654f9590655ca5a454a013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cf3f09ccf8b471298f86d9125a78c48"
I20260812 16:41:16.469050 6082 raft_consensus.cc:3037] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P 40ad1847223a42139c4cb6ced039231c [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:16.469074 6304 raft_consensus.cc:3037] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P 6cf3f09ccf8b471298f86d9125a78c48 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:16.469811 6082 raft_consensus.cc:2454] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P 40ad1847223a42139c4cb6ced039231c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3ebcb73a5654f9590655ca5a454a013 in term 1.
I20260812 16:41:16.469909 6304 raft_consensus.cc:2454] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P 6cf3f09ccf8b471298f86d9125a78c48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3ebcb73a5654f9590655ca5a454a013 in term 1.
I20260812 16:41:16.470016 6411 leader_election.cc:302] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P e3ebcb73a5654f9590655ca5a454a013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40ad1847223a42139c4cb6ced039231c, e3ebcb73a5654f9590655ca5a454a013; no voters:
I20260812 16:41:16.470131 6641 raft_consensus.cc:2781] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P e3ebcb73a5654f9590655ca5a454a013 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:41:16.470284 6641 raft_consensus.cc:686] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P e3ebcb73a5654f9590655ca5a454a013 [term 1 LEADER]: Becoming Leader. State: Replica: e3ebcb73a5654f9590655ca5a454a013, State: Running, Role: LEADER
I20260812 16:41:16.470383 6641 consensus_queue.cc:237] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P e3ebcb73a5654f9590655ca5a454a013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40ad1847223a42139c4cb6ced039231c" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 41699 } } peers { permanent_uuid: "e3ebcb73a5654f9590655ca5a454a013" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32809 } } peers { permanent_uuid: "6cf3f09ccf8b471298f86d9125a78c48" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37333 } }
I20260812 16:41:16.471331 5818 catalog_manager.cc:5495] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P e3ebcb73a5654f9590655ca5a454a013 reported cstate change: term changed from 0 to 1, leader changed from <none> to e3ebcb73a5654f9590655ca5a454a013 (127.0.253.195). New cstate: current_term: 1 leader_uuid: "e3ebcb73a5654f9590655ca5a454a013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40ad1847223a42139c4cb6ced039231c" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 41699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e3ebcb73a5654f9590655ca5a454a013" member_type: VOTER last_known_addr { host: "127.0.253.195" port: 32809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cf3f09ccf8b471298f86d9125a78c48" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37333 } health_report { overall_health: UNKNOWN } } }
W20260812 16:41:16.501755 6164 tablet.cc:2369] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P 40ad1847223a42139c4cb6ced039231c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 16:41:16.507220 6605 tablet.cc:2369] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P e3ebcb73a5654f9590655ca5a454a013: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:41:16.508523 6625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:16.506086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39852 (local address 127.0.253.195:32809)
0812 16:41:16.506155 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:16.506157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:16.506165 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:41:16.506206 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:16.506207 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:16.506225 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:16.506291 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:41:16.506393 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:16.506876 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:16.507801 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:16.508005 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:16.508063 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:16.508184 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:41:16.508285 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:41:16.508343 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:16.508376 (+ 33us) server_negotiation.cc:299] Negotiation successful
0812 16:41:16.508391 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 16:41:16.510360 6082 raft_consensus.cc:1261] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P 40ad1847223a42139c4cb6ced039231c [term 1 FOLLOWER]: Refusing update from remote peer e3ebcb73a5654f9590655ca5a454a013: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:41:16.510392 6304 raft_consensus.cc:1261] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P 6cf3f09ccf8b471298f86d9125a78c48 [term 1 FOLLOWER]: Refusing update from remote peer e3ebcb73a5654f9590655ca5a454a013: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:41:16.510754 6641 consensus_queue.cc:1035] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P e3ebcb73a5654f9590655ca5a454a013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40ad1847223a42139c4cb6ced039231c" member_type: VOTER last_known_addr { host: "127.0.253.193" port: 41699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:41:16.510917 6643 consensus_queue.cc:1035] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P e3ebcb73a5654f9590655ca5a454a013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cf3f09ccf8b471298f86d9125a78c48" member_type: VOTER last_known_addr { host: "127.0.253.194" port: 37333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:41:16.513293 6654 mvcc.cc:205] Tried to move back new op lower bound from 7317720582183546880 to 7317720582023442432. Current Snapshot: MvccSnapshot[applied={T|T < 7317720582183546880}]
I20260812 16:41:16.513652 6655 mvcc.cc:205] Tried to move back new op lower bound from 7317720582183546880 to 7317720582023442432. Current Snapshot: MvccSnapshot[applied={T|T < 7317720582183546880}]
I20260812 16:41:16.514376 6658 mvcc.cc:205] Tried to move back new op lower bound from 7317720582183546880 to 7317720582023442432. Current Snapshot: MvccSnapshot[applied={T|T < 7317720582183546880 or (T in {7317720582183546880})}]
I20260812 16:41:16.516395 5815 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:41:16.516491 5815 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:41:16.517400 5815 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0314014c83964e6092d1826632d2b3bc: Sending DeleteTablet for 3 replicas of tablet 030f6aebc2f04b648fbd6ba3cdf4bd5f
I20260812 16:41:16.517782 6031 tablet_service.cc:1514] Processing DeleteTablet for tablet 030f6aebc2f04b648fbd6ba3cdf4bd5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:41:16 UTC) from {username='slave'} at 127.0.0.1:59656
I20260812 16:41:16.517802 6488 tablet_service.cc:1514] Processing DeleteTablet for tablet 030f6aebc2f04b648fbd6ba3cdf4bd5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:41:16 UTC) from {username='slave'} at 127.0.0.1:39842
I20260812 16:41:16.517926 6265 tablet_service.cc:1514] Processing DeleteTablet for tablet 030f6aebc2f04b648fbd6ba3cdf4bd5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:41:16 UTC) from {username='slave'} at 127.0.0.1:49644
I20260812 16:41:16.518049 6675 tablet_replica.cc:330] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P e3ebcb73a5654f9590655ca5a454a013: stopping tablet replica
I20260812 16:41:16.518147 6675 raft_consensus.cc:2229] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P e3ebcb73a5654f9590655ca5a454a013 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:41:16.518182 6678 tablet_replica.cc:330] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P 6cf3f09ccf8b471298f86d9125a78c48: stopping tablet replica
I20260812 16:41:16.518285 6675 raft_consensus.cc:2258] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P e3ebcb73a5654f9590655ca5a454a013 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:41:16.518316 6678 raft_consensus.cc:2229] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P 6cf3f09ccf8b471298f86d9125a78c48 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:41:16.518451 6678 raft_consensus.cc:2258] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P 6cf3f09ccf8b471298f86d9125a78c48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:41:16.518648 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 5931
I20260812 16:41:16.518797 6675 ts_tablet_manager.cc:1905] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P e3ebcb73a5654f9590655ca5a454a013: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:41:16.518985 6678 ts_tablet_manager.cc:1905] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P 6cf3f09ccf8b471298f86d9125a78c48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:41:16.520890 6678 ts_tablet_manager.cc:1918] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P 6cf3f09ccf8b471298f86d9125a78c48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:41:16.521009 6678 log.cc:1196] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P 6cf3f09ccf8b471298f86d9125a78c48: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-1/wal/wals/030f6aebc2f04b648fbd6ba3cdf4bd5f
I20260812 16:41:16.521164 6675 ts_tablet_manager.cc:1918] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P e3ebcb73a5654f9590655ca5a454a013: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:41:16.521262 6675 log.cc:1196] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P e3ebcb73a5654f9590655ca5a454a013: Deleting WAL directory at /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552619305671-1015-0/minicluster-data/ts-2/wal/wals/030f6aebc2f04b648fbd6ba3cdf4bd5f
I20260812 16:41:16.521385 6678 ts_tablet_manager.cc:1939] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P 6cf3f09ccf8b471298f86d9125a78c48: Deleting consensus metadata
I20260812 16:41:16.521598 6675 ts_tablet_manager.cc:1939] T 030f6aebc2f04b648fbd6ba3cdf4bd5f P e3ebcb73a5654f9590655ca5a454a013: Deleting consensus metadata
I20260812 16:41:16.521924 5790 catalog_manager.cc:4841] TS 6cf3f09ccf8b471298f86d9125a78c48 (127.0.253.194:37333): tablet 030f6aebc2f04b648fbd6ba3cdf4bd5f (table test-table [id=1acffdd9279f4689b5401cdd505fff79]) successfully deleted
I20260812 16:41:16.522107 5793 catalog_manager.cc:4841] TS e3ebcb73a5654f9590655ca5a454a013 (127.0.253.195:32809): tablet 030f6aebc2f04b648fbd6ba3cdf4bd5f (table test-table [id=1acffdd9279f4689b5401cdd505fff79]) successfully deleted
W20260812 16:41:16.531222 5793 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.253.193:41699 (error 108)
I20260812 16:41:16.531474 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 6166
I20260812 16:41:16.534777 6624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:16.531708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.193:41699 (local address 127.0.0.1:59664)
0812 16:41:16.534612 (+ 2904us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:16.534725 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.193:41699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2867}
W20260812 16:41:16.561925 5793 catalog_manager.cc:4584] TS 40ad1847223a42139c4cb6ced039231c (127.0.253.193:41699): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 030f6aebc2f04b648fbd6ba3cdf4bd5f: Network error: Client connection negotiation failed: client connection to 127.0.253.193:41699: connect: Connection refused (error 111)
I20260812 16:41:16.567508 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 6385
I20260812 16:41:16.582768 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 5759
2026-08-12T16:41:16Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (919 ms)
[----------] 6 tests from EncryptionPolicyTest (9940 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:41:16 dist-test-slave-w4v5 krb5kdc[6686](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:41:16 dist-test-slave-w4v5 krb5kdc[6686](info): set up 2 sockets
Aug 12 16:41:16 dist-test-slave-w4v5 krb5kdc[6686](info): commencing operation
krb5kdc: starting...
W20260812 16:41:17.913807 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.308s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:41:18 dist-test-slave-w4v5 krb5kdc[6686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:41:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:41:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 16:41:18.044198 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37007
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37007
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:41:18.129418 6710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:18.129637 6710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:18.129658 6710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:18.130811 6710 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:41:18.130852 6710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:18.130867 6710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:18.130878 6710 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:41:18.130890 6710 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:41:18.132332 6710 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37007
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6710
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:18.132524 6710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:18.132685 6710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:18.137681 6720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:18.137677 6717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:18.137800 6716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:18.137916 6710 server_base.cc:1034] running on GCE node
I20260812 16:41:18.138207 6710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:18.138485 6710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:18.139672 6710 hybrid_clock.cc:648] HybridClock initialized: now 1786552878139635 us; error 66 us; skew 500 ppm
Aug 12 16:41:18 dist-test-slave-w4v5 krb5kdc[6686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552878, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:41:18.146575 6710 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 16:41:18.147179 6710 webserver.cc:460] Webserver started at http://127.0.0.1:42191/ using document root <none> and password file <none>
I20260812 16:41:18.147432 6710 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:18.147507 6710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:18.147707 6710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:18.148895 6710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "6e753561be88476ba7abcfd5749e7430"
format_stamp: "Formatted at 2026-08-12 16:41:18 on dist-test-slave-w4v5"
server_key: "f023b579e9ac14cd54b78f0357d6e7f4"
server_key_iv: "b8bdde853475d9ed686af2ffebe1cc62"
server_key_version: "encryptionkey@0"
I20260812 16:41:18.149313 6710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "6e753561be88476ba7abcfd5749e7430"
format_stamp: "Formatted at 2026-08-12 16:41:18 on dist-test-slave-w4v5"
server_key: "f023b579e9ac14cd54b78f0357d6e7f4"
server_key_iv: "b8bdde853475d9ed686af2ffebe1cc62"
server_key_version: "encryptionkey@0"
I20260812 16:41:18.150846 6710 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:41:18.151733 6738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:18.151924 6710 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:41:18.152050 6710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "6e753561be88476ba7abcfd5749e7430"
format_stamp: "Formatted at 2026-08-12 16:41:18 on dist-test-slave-w4v5"
server_key: "f023b579e9ac14cd54b78f0357d6e7f4"
server_key_iv: "b8bdde853475d9ed686af2ffebe1cc62"
server_key_version: "encryptionkey@0"
I20260812 16:41:18.152130 6710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:18.173722 6710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:18.176215 6710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:18.176393 6710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:18.181001 6710 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37007
I20260812 16:41:18.181046 6834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37007 every 8 connection(s)
I20260812 16:41:18.181463 6710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:41:18.182608 6836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:18.185000 6836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e753561be88476ba7abcfd5749e7430: Bootstrap starting.
I20260812 16:41:18.185680 6836 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e753561be88476ba7abcfd5749e7430: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:18.185997 6836 log.cc:824] T 00000000000000000000000000000000 P 6e753561be88476ba7abcfd5749e7430: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:18.186836 6836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e753561be88476ba7abcfd5749e7430: No bootstrap required, opened a new log
I20260812 16:41:18.188455 6836 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e753561be88476ba7abcfd5749e7430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e753561be88476ba7abcfd5749e7430" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37007 } }
I20260812 16:41:18.188603 6836 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e753561be88476ba7abcfd5749e7430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:18.188627 6836 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e753561be88476ba7abcfd5749e7430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e753561be88476ba7abcfd5749e7430, State: Initialized, Role: FOLLOWER
I20260812 16:41:18.188810 6836 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e753561be88476ba7abcfd5749e7430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e753561be88476ba7abcfd5749e7430" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37007 } }
I20260812 16:41:18.188876 6836 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e753561be88476ba7abcfd5749e7430 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:41:18.188918 6836 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e753561be88476ba7abcfd5749e7430 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:41:18.188967 6836 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e753561be88476ba7abcfd5749e7430 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:18.189534 6836 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e753561be88476ba7abcfd5749e7430 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e753561be88476ba7abcfd5749e7430" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37007 } }
I20260812 16:41:18.189647 6836 leader_election.cc:302] T 00000000000000000000000000000000 P 6e753561be88476ba7abcfd5749e7430 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e753561be88476ba7abcfd5749e7430; no voters:
I20260812 16:41:18.189836 6836 leader_election.cc:288] T 00000000000000000000000000000000 P 6e753561be88476ba7abcfd5749e7430 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:41:18.189916 6839 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e753561be88476ba7abcfd5749e7430 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:41:18.190095 6839 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e753561be88476ba7abcfd5749e7430 [term 1 LEADER]: Becoming Leader. State: Replica: 6e753561be88476ba7abcfd5749e7430, State: Running, Role: LEADER
I20260812 16:41:18.190106 6836 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e753561be88476ba7abcfd5749e7430 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:41:18.190305 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 6710
I20260812 16:41:18.190317 6839 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e753561be88476ba7abcfd5749e7430 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e753561be88476ba7abcfd5749e7430" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37007 } }
I20260812 16:41:18.190414 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:41:18.190650 1015 external_mini_cluster.cc:1428] Setting key da099f53c3863ee77e9da5297dfccdde
I20260812 16:41:18.190714 6841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e753561be88476ba7abcfd5749e7430 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e753561be88476ba7abcfd5749e7430. Latest consensus state: current_term: 1 leader_uuid: "6e753561be88476ba7abcfd5749e7430" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e753561be88476ba7abcfd5749e7430" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37007 } } }
I20260812 16:41:18.190793 6841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e753561be88476ba7abcfd5749e7430 [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:18.190865 6840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e753561be88476ba7abcfd5749e7430 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e753561be88476ba7abcfd5749e7430" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e753561be88476ba7abcfd5749e7430" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37007 } } }
I20260812 16:41:18.190927 6840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e753561be88476ba7abcfd5749e7430 [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:18.191022 6847 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:41:18.191597 6847 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:41:18.193166 6847 catalog_manager.cc:1347] Generated new cluster ID: 1422406a14e64aa4afde745df4ffab91
I20260812 16:41:18.193221 6847 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 16:41:18 dist-test-slave-w4v5 krb5kdc[6686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 16:41:18.202515 6847 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:41:18.203166 6847 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:41:18.206642 6862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:18.191368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41439 (local address 127.0.0.1:37007)
0812 16:41:18.191668 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:18.191694 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:18.195340 (+ 3646us) server_negotiation.cc:407] Connection header received
0812 16:41:18.195464 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:18.195471 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:18.195539 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:18.195680 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:18.196506 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:18.197908 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:18.199043 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:18.199671 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:18.199773 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:18.202796 (+ 3023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:18.202801 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:18.202801 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:18.202815 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:18.204903 (+ 2088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:18.205375 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:18.205378 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:18.205379 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:18.205486 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:18.205744 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:18.205745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:18.205745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:18.206013 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:18.206164 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:18.206307 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 16:41:18.206412 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":89,"threads_started":1}
I20260812 16:41:18.207659 1015 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:41:18.209246 6847 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e753561be88476ba7abcfd5749e7430: Generated new TSK 0
I20260812 16:41:18.209426 6847 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:41:18.215885 6862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:18.209330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40125 (local address 127.0.0.1:37007)
0812 16:41:18.209391 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:18.209393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:18.209821 (+ 428us) server_negotiation.cc:407] Connection header received
0812 16:41:18.209894 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:18.209896 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:18.209906 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:18.209961 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:18.210481 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:18.211012 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:18.211777 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:18.212023 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:18.212100 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:18.213308 (+ 1208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:18.213309 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:18.213310 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:18.213316 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:18.214609 (+ 1293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:18.215021 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:18.215022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:18.215023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:18.215117 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:18.215342 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:18.215343 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:18.215343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:18.215456 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:18.215587 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:18.215709 (+ 122us) server_negotiation.cc:299] Negotiation successful
0812 16:41:18.215779 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 16:41:18.217742 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 6710
W20260812 16:41:18.234545 6681 connection.cc:535] client connection to 127.0.0.1:37007 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37007): Connection reset by peer (error 104)
2026-08-12T16:41:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1642 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:41:18 dist-test-slave-w4v5 krb5kdc[6887](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:41:18 dist-test-slave-w4v5 krb5kdc[6887](info): set up 2 sockets
Aug 12 16:41:18 dist-test-slave-w4v5 krb5kdc[6887](info): commencing operation
krb5kdc: starting...
W20260812 16:41:19.544154 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.296s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:41:19 dist-test-slave-w4v5 krb5kdc[6887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:41:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:41:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 16:41:19.671725 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40523
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40523
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:41:19.759001 6908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:19.759356 6908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:19.759415 6908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:19.761533 6908 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:41:19.761659 6908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:19.761703 6908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:19.761726 6908 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:41:19.761754 6908 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:41:19.764338 6908 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40523
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:19.764643 6908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:19.767547 6908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:19.773420 6915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:19.773422 6917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:19.773640 6921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:19.773789 6908 server_base.cc:1034] running on GCE node
I20260812 16:41:19.774106 6908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:19.774504 6908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:19.775668 6908 hybrid_clock.cc:648] HybridClock initialized: now 1786552879775664 us; error 38 us; skew 500 ppm
Aug 12 16:41:19 dist-test-slave-w4v5 krb5kdc[6887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552879, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:41:19.778620 6908 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 16:41:19.780036 6908 webserver.cc:460] Webserver started at http://127.0.0.1:35757/ using document root <none> and password file <none>
I20260812 16:41:19.780284 6908 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:19.780360 6908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:19.780519 6908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:19.781932 6908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "0820136fc42a4317ac4bd3b16eeed090"
format_stamp: "Formatted at 2026-08-12 16:41:19 on dist-test-slave-w4v5"
server_key: "39a9750a6e59209ccbe66c19b6ee0e9d"
server_key_iv: "f85fb0e57f878f327c10d3e4fcbf14f3"
server_key_version: "encryptionkey@0"
I20260812 16:41:19.782419 6908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "0820136fc42a4317ac4bd3b16eeed090"
format_stamp: "Formatted at 2026-08-12 16:41:19 on dist-test-slave-w4v5"
server_key: "39a9750a6e59209ccbe66c19b6ee0e9d"
server_key_iv: "f85fb0e57f878f327c10d3e4fcbf14f3"
server_key_version: "encryptionkey@0"
I20260812 16:41:19.784149 6908 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:41:19.785017 6937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:19.785225 6908 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:41:19.785372 6908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "0820136fc42a4317ac4bd3b16eeed090"
format_stamp: "Formatted at 2026-08-12 16:41:19 on dist-test-slave-w4v5"
server_key: "39a9750a6e59209ccbe66c19b6ee0e9d"
server_key_iv: "f85fb0e57f878f327c10d3e4fcbf14f3"
server_key_version: "encryptionkey@0"
I20260812 16:41:19.785458 6908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:19.796578 6908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:19.799034 6908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:19.799228 6908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:19.803211 6908 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40523
I20260812 16:41:19.803267 7048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40523 every 8 connection(s)
I20260812 16:41:19.803628 6908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:41:19.804677 7051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:19.807057 7051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0820136fc42a4317ac4bd3b16eeed090: Bootstrap starting.
I20260812 16:41:19.807760 7051 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0820136fc42a4317ac4bd3b16eeed090: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:19.808059 7051 log.cc:824] T 00000000000000000000000000000000 P 0820136fc42a4317ac4bd3b16eeed090: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:19.808815 7051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0820136fc42a4317ac4bd3b16eeed090: No bootstrap required, opened a new log
I20260812 16:41:19.810423 7051 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0820136fc42a4317ac4bd3b16eeed090 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0820136fc42a4317ac4bd3b16eeed090" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40523 } }
I20260812 16:41:19.810578 7051 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0820136fc42a4317ac4bd3b16eeed090 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:19.810603 7051 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0820136fc42a4317ac4bd3b16eeed090 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0820136fc42a4317ac4bd3b16eeed090, State: Initialized, Role: FOLLOWER
I20260812 16:41:19.810808 7051 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0820136fc42a4317ac4bd3b16eeed090 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0820136fc42a4317ac4bd3b16eeed090" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40523 } }
I20260812 16:41:19.810876 7051 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0820136fc42a4317ac4bd3b16eeed090 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:41:19.810917 7051 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0820136fc42a4317ac4bd3b16eeed090 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:41:19.810968 7051 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0820136fc42a4317ac4bd3b16eeed090 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:19.811576 7051 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0820136fc42a4317ac4bd3b16eeed090 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0820136fc42a4317ac4bd3b16eeed090" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40523 } }
I20260812 16:41:19.811703 7051 leader_election.cc:302] T 00000000000000000000000000000000 P 0820136fc42a4317ac4bd3b16eeed090 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0820136fc42a4317ac4bd3b16eeed090; no voters:
I20260812 16:41:19.811945 7051 leader_election.cc:288] T 00000000000000000000000000000000 P 0820136fc42a4317ac4bd3b16eeed090 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:41:19.811967 7057 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0820136fc42a4317ac4bd3b16eeed090 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:41:19.811983 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 6908
I20260812 16:41:19.812098 7057 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0820136fc42a4317ac4bd3b16eeed090 [term 1 LEADER]: Becoming Leader. State: Replica: 0820136fc42a4317ac4bd3b16eeed090, State: Running, Role: LEADER
I20260812 16:41:19.812114 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:41:19.812186 7057 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0820136fc42a4317ac4bd3b16eeed090 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0820136fc42a4317ac4bd3b16eeed090" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40523 } }
I20260812 16:41:19.812224 7051 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0820136fc42a4317ac4bd3b16eeed090 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:41:19.812325 1015 external_mini_cluster.cc:1428] Setting key 13835f2044730ab6e1cc46339cc424b7
I20260812 16:41:19.812508 7057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0820136fc42a4317ac4bd3b16eeed090 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0820136fc42a4317ac4bd3b16eeed090. Latest consensus state: current_term: 1 leader_uuid: "0820136fc42a4317ac4bd3b16eeed090" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0820136fc42a4317ac4bd3b16eeed090" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40523 } } }
I20260812 16:41:19.812570 7057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0820136fc42a4317ac4bd3b16eeed090 [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:19.812681 7063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0820136fc42a4317ac4bd3b16eeed090 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0820136fc42a4317ac4bd3b16eeed090" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0820136fc42a4317ac4bd3b16eeed090" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40523 } } }
I20260812 16:41:19.812774 7063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0820136fc42a4317ac4bd3b16eeed090 [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:19.812891 7068 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:41:19.813501 7068 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:41:19.815289 7068 catalog_manager.cc:1347] Generated new cluster ID: 287e01ce9fc140f0b26df1d77f0e4359
I20260812 16:41:19.815343 7068 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:41:19.819048 7076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:19.813004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59729 (local address 127.0.0.1:40523)
0812 16:41:19.813470 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:19.813480 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:19.813514 (+ 34us) server_negotiation.cc:407] Connection header received
0812 16:41:19.813642 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:19.813648 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:19.813756 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:19.813914 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:19.814154 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:19.815584 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:19.816708 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:19.817388 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:19.817511 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:19.818287 (+ 776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:19.818292 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:19.818292 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:41:19.818528 (+ 236us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:19.818573 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:41:19.818589 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:19.818684 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:19.818779 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 16:41:19.818813 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":90,"threads_started":1}
I20260812 16:41:19.820133 1015 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:41:19.825349 7076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:19.821735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43779 (local address 127.0.0.1:40523)
0812 16:41:19.821778 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:19.821780 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:19.821885 (+ 105us) server_negotiation.cc:407] Connection header received
0812 16:41:19.821917 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:19.821918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:19.821944 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:19.822021 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:19.822162 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:19.822929 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:19.823930 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:19.824289 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:19.824367 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:19.825080 (+ 713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:19.825081 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:19.825082 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:41:19.825087 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:19.825103 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:41:19.825108 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:19.825183 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:19.825254 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 16:41:19.825267 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 16:41:19.845073 7068 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:41:19.846061 7068 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:41:19.851040 7068 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0820136fc42a4317ac4bd3b16eeed090: Generated new TSK 0
I20260812 16:41:19.851362 7068 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:41:19.854568 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 6908
W20260812 16:41:19.871063 6885 connection.cc:535] client connection to 127.0.0.1:40523 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40523): Connection reset by peer (error 104)
2026-08-12T16:41:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1636 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:41:19 dist-test-slave-w4v5 krb5kdc[7100](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:41:19 dist-test-slave-w4v5 krb5kdc[7100](info): set up 2 sockets
Aug 12 16:41:19 dist-test-slave-w4v5 krb5kdc[7100](info): commencing operation
krb5kdc: starting...
W20260812 16:41:21.185901 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.301s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:41:21 dist-test-slave-w4v5 krb5kdc[7100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:41:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:41:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 16:41:21.316253 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39757
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39757
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:41:21.401007 7121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:21.401274 7121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:21.401319 7121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:21.402536 7121 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:41:21.402619 7121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:21.402652 7121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:21.402673 7121 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:41:21.402702 7121 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:41:21.404294 7121 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39757
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:21.404582 7121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:21.404839 7121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:21.409961 7127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:21.409948 7131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:21.409977 7129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:21.410193 7121 server_base.cc:1034] running on GCE node
I20260812 16:41:21.410439 7121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:21.410702 7121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:21.411880 7121 hybrid_clock.cc:648] HybridClock initialized: now 1786552881411855 us; error 48 us; skew 500 ppm
Aug 12 16:41:21 dist-test-slave-w4v5 krb5kdc[7100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552881, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:41:21.414395 7121 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 16:41:21.414863 7121 webserver.cc:460] Webserver started at http://127.0.0.1:33993/ using document root <none> and password file <none>
I20260812 16:41:21.415191 7121 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:21.415266 7121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:21.415406 7121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:21.416540 7121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "d712d5d67cfd436190ec1e9531a29a47"
format_stamp: "Formatted at 2026-08-12 16:41:21 on dist-test-slave-w4v5"
server_key: "114a00d34d36d5d9ee40224cd998c257"
server_key_iv: "0ed8bf25b1216a47925e56b9ea774ae1"
server_key_version: "encryptionkey@0"
I20260812 16:41:21.416935 7121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "d712d5d67cfd436190ec1e9531a29a47"
format_stamp: "Formatted at 2026-08-12 16:41:21 on dist-test-slave-w4v5"
server_key: "114a00d34d36d5d9ee40224cd998c257"
server_key_iv: "0ed8bf25b1216a47925e56b9ea774ae1"
server_key_version: "encryptionkey@0"
I20260812 16:41:21.418342 7121 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:41:21.419359 7141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:21.419575 7121 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:41:21.419685 7121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "d712d5d67cfd436190ec1e9531a29a47"
format_stamp: "Formatted at 2026-08-12 16:41:21 on dist-test-slave-w4v5"
server_key: "114a00d34d36d5d9ee40224cd998c257"
server_key_iv: "0ed8bf25b1216a47925e56b9ea774ae1"
server_key_version: "encryptionkey@0"
I20260812 16:41:21.419793 7121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:21.423321 7121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:21.425199 7121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:21.425364 7121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:21.429742 7121 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39757
I20260812 16:41:21.429769 7240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39757 every 8 connection(s)
I20260812 16:41:21.430115 7121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:41:21.431272 7242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:21.431666 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 7121
I20260812 16:41:21.431793 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:41:21.432006 1015 external_mini_cluster.cc:1428] Setting key 3b602af9671cfff3c46a0866f3b2e87d
I20260812 16:41:21.434558 7242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d712d5d67cfd436190ec1e9531a29a47: Bootstrap starting.
I20260812 16:41:21.434628 7248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:21.432541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53651 (local address 127.0.0.1:39757)
0812 16:41:21.432747 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:21.432758 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:21.432793 (+ 35us) server_negotiation.cc:407] Connection header received
0812 16:41:21.433345 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:21.433355 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:21.433391 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:21.433547 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:21.434201 (+ 654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:21.434205 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:21.434205 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:41:21.434209 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:21.434252 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:41:21.434266 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:21.434339 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:21.434403 (+ 64us) server_negotiation.cc:299] Negotiation successful
0812 16:41:21.434443 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":94,"threads_started":1}
I20260812 16:41:21.435330 7242 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d712d5d67cfd436190ec1e9531a29a47: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:21.435647 1015 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:41:21.435670 7242 log.cc:824] T 00000000000000000000000000000000 P d712d5d67cfd436190ec1e9531a29a47: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:21.436411 7242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d712d5d67cfd436190ec1e9531a29a47: No bootstrap required, opened a new log
I20260812 16:41:21.438083 7248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:21.437087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46485 (local address 127.0.0.1:39757)
0812 16:41:21.437123 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:21.437124 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:21.437236 (+ 112us) server_negotiation.cc:407] Connection header received
0812 16:41:21.437303 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:21.437304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:21.437307 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:21.437350 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:21.437872 (+ 522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:21.437873 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:21.437873 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:41:21.437877 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:21.437889 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:41:21.437893 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:21.437937 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:21.437992 (+ 55us) server_negotiation.cc:299] Negotiation successful
0812 16:41:21.438003 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 16:41:21.439064 7242 raft_consensus.cc:348] T 00000000000000000000000000000000 P d712d5d67cfd436190ec1e9531a29a47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d712d5d67cfd436190ec1e9531a29a47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39757 } }
I20260812 16:41:21.439287 7242 raft_consensus.cc:374] T 00000000000000000000000000000000 P d712d5d67cfd436190ec1e9531a29a47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:21.439334 7242 raft_consensus.cc:729] T 00000000000000000000000000000000 P d712d5d67cfd436190ec1e9531a29a47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d712d5d67cfd436190ec1e9531a29a47, State: Initialized, Role: FOLLOWER
I20260812 16:41:21.439463 7242 consensus_queue.cc:260] T 00000000000000000000000000000000 P d712d5d67cfd436190ec1e9531a29a47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d712d5d67cfd436190ec1e9531a29a47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39757 } }
I20260812 16:41:21.439564 7242 raft_consensus.cc:388] T 00000000000000000000000000000000 P d712d5d67cfd436190ec1e9531a29a47 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:41:21.439611 7242 raft_consensus.cc:482] T 00000000000000000000000000000000 P d712d5d67cfd436190ec1e9531a29a47 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:41:21.439663 7242 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d712d5d67cfd436190ec1e9531a29a47 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:21.440263 7242 raft_consensus.cc:504] T 00000000000000000000000000000000 P d712d5d67cfd436190ec1e9531a29a47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d712d5d67cfd436190ec1e9531a29a47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39757 } }
I20260812 16:41:21.440383 7242 leader_election.cc:302] T 00000000000000000000000000000000 P d712d5d67cfd436190ec1e9531a29a47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d712d5d67cfd436190ec1e9531a29a47; no voters:
I20260812 16:41:21.440587 7242 leader_election.cc:288] T 00000000000000000000000000000000 P d712d5d67cfd436190ec1e9531a29a47 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:41:21.440653 7261 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d712d5d67cfd436190ec1e9531a29a47 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:41:21.440837 7242 sys_catalog.cc:564] T 00000000000000000000000000000000 P d712d5d67cfd436190ec1e9531a29a47 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:41:21.440927 7261 raft_consensus.cc:686] T 00000000000000000000000000000000 P d712d5d67cfd436190ec1e9531a29a47 [term 1 LEADER]: Becoming Leader. State: Replica: d712d5d67cfd436190ec1e9531a29a47, State: Running, Role: LEADER
I20260812 16:41:21.441039 7261 consensus_queue.cc:237] T 00000000000000000000000000000000 P d712d5d67cfd436190ec1e9531a29a47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d712d5d67cfd436190ec1e9531a29a47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39757 } }
I20260812 16:41:21.441430 7262 sys_catalog.cc:455] T 00000000000000000000000000000000 P d712d5d67cfd436190ec1e9531a29a47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d712d5d67cfd436190ec1e9531a29a47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d712d5d67cfd436190ec1e9531a29a47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39757 } } }
I20260812 16:41:21.441533 7262 sys_catalog.cc:458] T 00000000000000000000000000000000 P d712d5d67cfd436190ec1e9531a29a47 [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:21.441448 7264 sys_catalog.cc:455] T 00000000000000000000000000000000 P d712d5d67cfd436190ec1e9531a29a47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d712d5d67cfd436190ec1e9531a29a47. Latest consensus state: current_term: 1 leader_uuid: "d712d5d67cfd436190ec1e9531a29a47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d712d5d67cfd436190ec1e9531a29a47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39757 } } }
I20260812 16:41:21.441751 7264 sys_catalog.cc:458] T 00000000000000000000000000000000 P d712d5d67cfd436190ec1e9531a29a47 [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:21.441764 7276 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:41:21.442279 7276 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:41:21.443940 7276 catalog_manager.cc:1347] Generated new cluster ID: 4fb90786e2b248a6961d1e514a911ba9
I20260812 16:41:21.444005 7276 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:41:21.470822 7276 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:41:21.471056 7276 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:41:21.474715 7276 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d712d5d67cfd436190ec1e9531a29a47: Generated new TSK 0
I20260812 16:41:21.474918 7276 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:41:21.482995 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 7121
2026-08-12T16:41:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1624 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:41:21 dist-test-slave-w4v5 krb5kdc[7296](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:41:21 dist-test-slave-w4v5 krb5kdc[7296](info): set up 2 sockets
Aug 12 16:41:21 dist-test-slave-w4v5 krb5kdc[7296](info): commencing operation
krb5kdc: starting...
W20260812 16:41:22.804275 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.295s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:41:22 dist-test-slave-w4v5 krb5kdc[7296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:41:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:41:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 16:41:22.928212 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39813
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39813
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:41:23.012223 7314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:23.012456 7314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:23.012481 7314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:23.013597 7314 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:41:23.013633 7314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:23.013654 7314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:23.013667 7314 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:41:23.013680 7314 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:41:23.015149 7314 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39813
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7314
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:23.015364 7314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:23.015538 7314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:23.020794 7320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:23.020790 7324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:23.020789 7322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:23.021145 7314 server_base.cc:1034] running on GCE node
I20260812 16:41:23.021427 7314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:23.021744 7314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:23.022946 7314 hybrid_clock.cc:648] HybridClock initialized: now 1786552883022913 us; error 47 us; skew 500 ppm
Aug 12 16:41:23 dist-test-slave-w4v5 krb5kdc[7296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552883, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:41:23.026008 7314 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 16:41:23.026441 7314 webserver.cc:460] Webserver started at http://127.0.0.1:34263/ using document root <none> and password file <none>
I20260812 16:41:23.026666 7314 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:23.026713 7314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:23.026809 7314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:23.027981 7314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "00ca6eebd8ff4931a46c8f0b994b9efe"
format_stamp: "Formatted at 2026-08-12 16:41:23 on dist-test-slave-w4v5"
server_key: "0fbcd2bb8b8e8725c7d3fdce0d40972b"
server_key_iv: "fc2758be3aec5158b51a0543b0a526cd"
server_key_version: "encryptionkey@0"
I20260812 16:41:23.028302 7314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "00ca6eebd8ff4931a46c8f0b994b9efe"
format_stamp: "Formatted at 2026-08-12 16:41:23 on dist-test-slave-w4v5"
server_key: "0fbcd2bb8b8e8725c7d3fdce0d40972b"
server_key_iv: "fc2758be3aec5158b51a0543b0a526cd"
server_key_version: "encryptionkey@0"
I20260812 16:41:23.029726 7314 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:41:23.030606 7335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:23.030841 7314 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:41:23.030941 7314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "00ca6eebd8ff4931a46c8f0b994b9efe"
format_stamp: "Formatted at 2026-08-12 16:41:23 on dist-test-slave-w4v5"
server_key: "0fbcd2bb8b8e8725c7d3fdce0d40972b"
server_key_iv: "fc2758be3aec5158b51a0543b0a526cd"
server_key_version: "encryptionkey@0"
I20260812 16:41:23.031028 7314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:23.044687 7314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:23.047261 7314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:23.047387 7314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:23.051764 7314 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39813
I20260812 16:41:23.051802 7431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39813 every 8 connection(s)
I20260812 16:41:23.052194 7314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:41:23.053308 7436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:23.053768 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 7314
I20260812 16:41:23.053876 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:41:23.054107 1015 external_mini_cluster.cc:1428] Setting key 2596f891a1a4ad0fedf9d7e4276abd01
I20260812 16:41:23.056607 7436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00ca6eebd8ff4931a46c8f0b994b9efe: Bootstrap starting.
I20260812 16:41:23.057349 7436 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00ca6eebd8ff4931a46c8f0b994b9efe: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:23.057660 7436 log.cc:824] T 00000000000000000000000000000000 P 00ca6eebd8ff4931a46c8f0b994b9efe: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:23.058378 7436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00ca6eebd8ff4931a46c8f0b994b9efe: No bootstrap required, opened a new log
I20260812 16:41:23.060297 7436 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00ca6eebd8ff4931a46c8f0b994b9efe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ca6eebd8ff4931a46c8f0b994b9efe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39813 } }
I20260812 16:41:23.060433 7436 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00ca6eebd8ff4931a46c8f0b994b9efe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:23.060456 7436 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00ca6eebd8ff4931a46c8f0b994b9efe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00ca6eebd8ff4931a46c8f0b994b9efe, State: Initialized, Role: FOLLOWER
I20260812 16:41:23.060599 7436 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00ca6eebd8ff4931a46c8f0b994b9efe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ca6eebd8ff4931a46c8f0b994b9efe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39813 } }
I20260812 16:41:23.060673 7436 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00ca6eebd8ff4931a46c8f0b994b9efe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:41:23.060695 7436 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00ca6eebd8ff4931a46c8f0b994b9efe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:41:23.060765 7436 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00ca6eebd8ff4931a46c8f0b994b9efe [term 0 FOLLOWER]: Advancing to term 1
Aug 12 16:41:23 dist-test-slave-w4v5 krb5kdc[7296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 16:41:23.061483 7436 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00ca6eebd8ff4931a46c8f0b994b9efe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ca6eebd8ff4931a46c8f0b994b9efe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39813 } }
I20260812 16:41:23.061582 7436 leader_election.cc:302] T 00000000000000000000000000000000 P 00ca6eebd8ff4931a46c8f0b994b9efe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00ca6eebd8ff4931a46c8f0b994b9efe; no voters:
I20260812 16:41:23.061794 7436 leader_election.cc:288] T 00000000000000000000000000000000 P 00ca6eebd8ff4931a46c8f0b994b9efe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:41:23.061832 7443 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00ca6eebd8ff4931a46c8f0b994b9efe [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:41:23.062026 7443 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00ca6eebd8ff4931a46c8f0b994b9efe [term 1 LEADER]: Becoming Leader. State: Replica: 00ca6eebd8ff4931a46c8f0b994b9efe, State: Running, Role: LEADER
I20260812 16:41:23.062071 7436 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00ca6eebd8ff4931a46c8f0b994b9efe [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:41:23.062206 7443 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00ca6eebd8ff4931a46c8f0b994b9efe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ca6eebd8ff4931a46c8f0b994b9efe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39813 } }
I20260812 16:41:23.062616 7444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00ca6eebd8ff4931a46c8f0b994b9efe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00ca6eebd8ff4931a46c8f0b994b9efe. Latest consensus state: current_term: 1 leader_uuid: "00ca6eebd8ff4931a46c8f0b994b9efe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ca6eebd8ff4931a46c8f0b994b9efe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39813 } } }
I20260812 16:41:23.062700 7444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00ca6eebd8ff4931a46c8f0b994b9efe [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:23.062774 7446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00ca6eebd8ff4931a46c8f0b994b9efe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00ca6eebd8ff4931a46c8f0b994b9efe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ca6eebd8ff4931a46c8f0b994b9efe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39813 } } }
I20260812 16:41:23.062839 7446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00ca6eebd8ff4931a46c8f0b994b9efe [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:23.063802 7454 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:41:23.064366 7454 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:41:23.066078 7454 catalog_manager.cc:1347] Generated new cluster ID: c46eb0806e1a4cad8308514e98e03001
I20260812 16:41:23.066128 7440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:23.054713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34821 (local address 127.0.0.1:39813)
0812 16:41:23.054924 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:23.054935 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:23.054972 (+ 37us) server_negotiation.cc:407] Connection header received
0812 16:41:23.055547 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:23.055557 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:23.055622 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:23.055792 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:23.056407 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:23.057799 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:23.058635 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:23.059166 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:23.059290 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:23.062107 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:23.062111 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:23.062112 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:23.062125 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:23.064380 (+ 2255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:23.064814 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:23.064818 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:23.064819 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:23.064930 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:23.065169 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:23.065170 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:23.065170 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:23.065481 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:23.065645 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:23.065780 (+ 135us) server_negotiation.cc:299] Negotiation successful
0812 16:41:23.065928 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":93,"threads_started":1}
I20260812 16:41:23.066180 7454 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:41:23.067129 1015 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:41:23.074756 7440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:23.068550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59161 (local address 127.0.0.1:39813)
0812 16:41:23.068589 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:23.068591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:23.068657 (+ 66us) server_negotiation.cc:407] Connection header received
0812 16:41:23.068730 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:23.068731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:23.068740 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:23.068798 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:23.069331 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:23.069817 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:23.070558 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:23.070799 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:23.070877 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:23.072127 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:23.072128 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:23.072129 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:23.072136 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:23.073587 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:23.073967 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:23.073968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:23.073969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:23.074026 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:23.074234 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:23.074235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:23.074236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:23.074348 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:23.074451 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:23.074560 (+ 109us) server_negotiation.cc:299] Negotiation successful
0812 16:41:23.074649 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 16:41:23.078950 7454 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:41:23.079506 7454 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:41:23.085166 7454 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00ca6eebd8ff4931a46c8f0b994b9efe: Generated new TSK 0
I20260812 16:41:23.085384 7454 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:41:23.090085 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 7314
2026-08-12T16:41:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1609 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:41:23 dist-test-slave-w4v5 krb5kdc[7483](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:41:23 dist-test-slave-w4v5 krb5kdc[7483](info): set up 2 sockets
Aug 12 16:41:23 dist-test-slave-w4v5 krb5kdc[7483](info): commencing operation
krb5kdc: starting...
W20260812 16:41:24.416922 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.298s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:41:24 dist-test-slave-w4v5 krb5kdc[7483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:41:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:41:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 16:41:24.546267 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45091
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:41:24.634698 7500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:24.634976 7500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:24.635028 7500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:24.636334 7500 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:41:24.636415 7500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:24.636442 7500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:24.636469 7500 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:41:24.636533 7500 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:41:24.638072 7500 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:24.638362 7500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:24.638612 7500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:24.643714 7517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:24.643739 7515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:24.643710 7514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:24.643908 7500 server_base.cc:1034] running on GCE node
I20260812 16:41:24.644203 7500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:24.644500 7500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:24.645675 7500 hybrid_clock.cc:648] HybridClock initialized: now 1786552884645656 us; error 49 us; skew 500 ppm
Aug 12 16:41:24 dist-test-slave-w4v5 krb5kdc[7483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552884, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:41:24.652408 7500 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 16:41:24.652868 7500 webserver.cc:460] Webserver started at http://127.0.0.1:42171/ using document root <none> and password file <none>
I20260812 16:41:24.653107 7500 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:24.653180 7500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:24.653337 7500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:24.654433 7500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "8730fba98aad4587a0e8d814ed1e4ae8"
format_stamp: "Formatted at 2026-08-12 16:41:24 on dist-test-slave-w4v5"
server_key: "8b7a10556746db1ebecc0d63bb0526a6"
server_key_iv: "9177dfb4b33db7cf367da61c72bd57ce"
server_key_version: "encryptionkey@0"
I20260812 16:41:24.654831 7500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "8730fba98aad4587a0e8d814ed1e4ae8"
format_stamp: "Formatted at 2026-08-12 16:41:24 on dist-test-slave-w4v5"
server_key: "8b7a10556746db1ebecc0d63bb0526a6"
server_key_iv: "9177dfb4b33db7cf367da61c72bd57ce"
server_key_version: "encryptionkey@0"
I20260812 16:41:24.656337 7500 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260812 16:41:24.657187 7525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:24.657378 7500 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:41:24.657486 7500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "8730fba98aad4587a0e8d814ed1e4ae8"
format_stamp: "Formatted at 2026-08-12 16:41:24 on dist-test-slave-w4v5"
server_key: "8b7a10556746db1ebecc0d63bb0526a6"
server_key_iv: "9177dfb4b33db7cf367da61c72bd57ce"
server_key_version: "encryptionkey@0"
I20260812 16:41:24.657609 7500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:24.673532 7500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:24.675978 7500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:24.676148 7500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:24.680449 7500 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45091
I20260812 16:41:24.680490 7621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45091 every 8 connection(s)
I20260812 16:41:24.680879 7500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:41:24.681912 7622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:24.682296 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 7500
I20260812 16:41:24.682435 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:41:24.682653 1015 external_mini_cluster.cc:1428] Setting key a1503a7f4d6cf13494e62749912f0c8c
I20260812 16:41:24.685057 7622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8730fba98aad4587a0e8d814ed1e4ae8: Bootstrap starting.
I20260812 16:41:24.685827 7622 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8730fba98aad4587a0e8d814ed1e4ae8: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:24.686149 7622 log.cc:824] T 00000000000000000000000000000000 P 8730fba98aad4587a0e8d814ed1e4ae8: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:24.686918 7622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8730fba98aad4587a0e8d814ed1e4ae8: No bootstrap required, opened a new log
I20260812 16:41:24.688512 7627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:24.683244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39315 (local address 127.0.0.1:45091)
0812 16:41:24.683434 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:24.683443 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:24.683487 (+ 44us) server_negotiation.cc:407] Connection header received
0812 16:41:24.683929 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:24.683936 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:24.683981 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:24.684107 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:24.684359 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:24.685273 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:24.686424 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:24.686970 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:24.687131 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:24.687890 (+ 759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:24.687894 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:24.687894 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:41:24.688063 (+ 169us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:24.688093 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:41:24.688105 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:24.688188 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:24.688268 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 16:41:24.688334 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":96,"threads_started":1}
I20260812 16:41:24.688988 7622 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8730fba98aad4587a0e8d814ed1e4ae8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8730fba98aad4587a0e8d814ed1e4ae8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45091 } }
I20260812 16:41:24.689203 7622 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8730fba98aad4587a0e8d814ed1e4ae8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:24.689258 7622 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8730fba98aad4587a0e8d814ed1e4ae8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8730fba98aad4587a0e8d814ed1e4ae8, State: Initialized, Role: FOLLOWER
I20260812 16:41:24.689400 7622 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8730fba98aad4587a0e8d814ed1e4ae8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8730fba98aad4587a0e8d814ed1e4ae8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45091 } }
I20260812 16:41:24.689520 7622 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8730fba98aad4587a0e8d814ed1e4ae8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:41:24.689558 7622 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8730fba98aad4587a0e8d814ed1e4ae8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:41:24.689579 1015 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:41:24.689612 7622 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8730fba98aad4587a0e8d814ed1e4ae8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:24.690151 7622 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8730fba98aad4587a0e8d814ed1e4ae8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8730fba98aad4587a0e8d814ed1e4ae8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45091 } }
I20260812 16:41:24.690285 7622 leader_election.cc:302] T 00000000000000000000000000000000 P 8730fba98aad4587a0e8d814ed1e4ae8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8730fba98aad4587a0e8d814ed1e4ae8; no voters:
I20260812 16:41:24.690460 7622 leader_election.cc:288] T 00000000000000000000000000000000 P 8730fba98aad4587a0e8d814ed1e4ae8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:41:24.690513 7630 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8730fba98aad4587a0e8d814ed1e4ae8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:41:24.690642 7630 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8730fba98aad4587a0e8d814ed1e4ae8 [term 1 LEADER]: Becoming Leader. State: Replica: 8730fba98aad4587a0e8d814ed1e4ae8, State: Running, Role: LEADER
I20260812 16:41:24.690748 7622 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8730fba98aad4587a0e8d814ed1e4ae8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:41:24.690737 7630 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8730fba98aad4587a0e8d814ed1e4ae8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8730fba98aad4587a0e8d814ed1e4ae8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45091 } }
I20260812 16:41:24.691241 7630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8730fba98aad4587a0e8d814ed1e4ae8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8730fba98aad4587a0e8d814ed1e4ae8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8730fba98aad4587a0e8d814ed1e4ae8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45091 } } }
I20260812 16:41:24.691279 7637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8730fba98aad4587a0e8d814ed1e4ae8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8730fba98aad4587a0e8d814ed1e4ae8. Latest consensus state: current_term: 1 leader_uuid: "8730fba98aad4587a0e8d814ed1e4ae8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8730fba98aad4587a0e8d814ed1e4ae8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45091 } } }
I20260812 16:41:24.691412 7630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8730fba98aad4587a0e8d814ed1e4ae8 [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:24.691856 7637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8730fba98aad4587a0e8d814ed1e4ae8 [sys.catalog]: This master's current role is: LEADER
W20260812 16:41:24.692746 7664 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8730fba98aad4587a0e8d814ed1e4ae8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 16:41:24.692855 7664 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:41:24.692947 7655 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:41:24.693106 7655 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:41:24.694555 7627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:24.691120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56093 (local address 127.0.0.1:45091)
0812 16:41:24.691160 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:24.691162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:24.691654 (+ 492us) server_negotiation.cc:407] Connection header received
0812 16:41:24.691756 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:24.691757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:24.691769 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:24.691818 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:24.692003 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:24.692600 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:24.693360 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:24.693568 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:24.693643 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:24.694319 (+ 676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:24.694321 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:24.694321 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:41:24.694325 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:24.694338 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:41:24.694342 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:24.694392 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:24.694456 (+ 64us) server_negotiation.cc:299] Negotiation successful
0812 16:41:24.694479 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 16:41:24.695518 7655 catalog_manager.cc:1347] Generated new cluster ID: 7106ba7329ee490d83fd41c9ba1b438c
I20260812 16:41:24.695583 7655 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:41:24.709750 7655 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:41:24.710283 7655 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:41:24.719498 7655 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8730fba98aad4587a0e8d814ed1e4ae8: Generated new TSK 0
I20260812 16:41:24.719781 7655 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:41:24.724962 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 7500
2026-08-12T16:41:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1634 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:41:24 dist-test-slave-w4v5 krb5kdc[7675](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:41:24 dist-test-slave-w4v5 krb5kdc[7675](info): set up 2 sockets
Aug 12 16:41:24 dist-test-slave-w4v5 krb5kdc[7675](info): commencing operation
krb5kdc: starting...
W20260812 16:41:26.041726 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.288s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:41:26 dist-test-slave-w4v5 krb5kdc[7675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552886, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:41:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:41:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 16:41:26.170095 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35843
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35843
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:41:26.256227 7694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:26.256470 7694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:26.256510 7694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:26.257686 7694 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:41:26.257753 7694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:26.257776 7694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:26.257798 7694 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:41:26.257828 7694 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:41:26.259351 7694 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35843
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:26.259639 7694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:26.259871 7694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:26.265005 7701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:26.265017 7704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:26.265018 7700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:26.265319 7694 server_base.cc:1034] running on GCE node
I20260812 16:41:26.265618 7694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:26.265923 7694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:26.267146 7694 hybrid_clock.cc:648] HybridClock initialized: now 1786552886267083 us; error 76 us; skew 500 ppm
Aug 12 16:41:26 dist-test-slave-w4v5 krb5kdc[7675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552886, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:41:26.269788 7694 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 16:41:26.270252 7694 webserver.cc:460] Webserver started at http://127.0.0.1:39997/ using document root <none> and password file <none>
I20260812 16:41:26.270502 7694 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:26.270576 7694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:26.270735 7694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:26.271900 7694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "9b983e73790c48be916618551f4d375c"
format_stamp: "Formatted at 2026-08-12 16:41:26 on dist-test-slave-w4v5"
server_key: "d07a5dc1d6401e25dab979bf7cfdf166"
server_key_iv: "b6df1fb804d18f66b61a6e329c0cb5c4"
server_key_version: "encryptionkey@0"
I20260812 16:41:26.272289 7694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "9b983e73790c48be916618551f4d375c"
format_stamp: "Formatted at 2026-08-12 16:41:26 on dist-test-slave-w4v5"
server_key: "d07a5dc1d6401e25dab979bf7cfdf166"
server_key_iv: "b6df1fb804d18f66b61a6e329c0cb5c4"
server_key_version: "encryptionkey@0"
I20260812 16:41:26.273731 7694 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:41:26.274585 7720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:26.274808 7694 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:41:26.274931 7694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "9b983e73790c48be916618551f4d375c"
format_stamp: "Formatted at 2026-08-12 16:41:26 on dist-test-slave-w4v5"
server_key: "d07a5dc1d6401e25dab979bf7cfdf166"
server_key_iv: "b6df1fb804d18f66b61a6e329c0cb5c4"
server_key_version: "encryptionkey@0"
I20260812 16:41:26.275010 7694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:26.278450 7694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:26.280597 7694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:26.280764 7694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:26.285367 7694 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35843
I20260812 16:41:26.285396 7811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35843 every 8 connection(s)
I20260812 16:41:26.285755 7694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:41:26.286885 7812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:26.289324 7812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b983e73790c48be916618551f4d375c: Bootstrap starting.
I20260812 16:41:26.289934 7812 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b983e73790c48be916618551f4d375c: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:26.290234 7812 log.cc:824] T 00000000000000000000000000000000 P 9b983e73790c48be916618551f4d375c: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:26.291013 7812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b983e73790c48be916618551f4d375c: No bootstrap required, opened a new log
I20260812 16:41:26.292678 7812 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b983e73790c48be916618551f4d375c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b983e73790c48be916618551f4d375c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35843 } }
I20260812 16:41:26.292833 7812 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b983e73790c48be916618551f4d375c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:26.292857 7812 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b983e73790c48be916618551f4d375c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b983e73790c48be916618551f4d375c, State: Initialized, Role: FOLLOWER
I20260812 16:41:26.293021 7812 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b983e73790c48be916618551f4d375c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b983e73790c48be916618551f4d375c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35843 } }
I20260812 16:41:26.293087 7812 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b983e73790c48be916618551f4d375c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:41:26.293109 7812 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b983e73790c48be916618551f4d375c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:41:26.293176 7812 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b983e73790c48be916618551f4d375c [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:26.293790 7812 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b983e73790c48be916618551f4d375c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b983e73790c48be916618551f4d375c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35843 } }
I20260812 16:41:26.293882 7812 leader_election.cc:302] T 00000000000000000000000000000000 P 9b983e73790c48be916618551f4d375c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b983e73790c48be916618551f4d375c; no voters:
I20260812 16:41:26.294099 7812 leader_election.cc:288] T 00000000000000000000000000000000 P 9b983e73790c48be916618551f4d375c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:41:26.294137 7816 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b983e73790c48be916618551f4d375c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:41:26.294340 7816 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b983e73790c48be916618551f4d375c [term 1 LEADER]: Becoming Leader. State: Replica: 9b983e73790c48be916618551f4d375c, State: Running, Role: LEADER
I20260812 16:41:26.294445 7812 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b983e73790c48be916618551f4d375c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:41:26.294525 7816 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b983e73790c48be916618551f4d375c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b983e73790c48be916618551f4d375c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35843 } }
I20260812 16:41:26.294987 7818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b983e73790c48be916618551f4d375c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b983e73790c48be916618551f4d375c. Latest consensus state: current_term: 1 leader_uuid: "9b983e73790c48be916618551f4d375c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b983e73790c48be916618551f4d375c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35843 } } }
I20260812 16:41:26.295159 7818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b983e73790c48be916618551f4d375c [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:26.294984 7817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b983e73790c48be916618551f4d375c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b983e73790c48be916618551f4d375c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b983e73790c48be916618551f4d375c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35843 } } }
I20260812 16:41:26.295388 7817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b983e73790c48be916618551f4d375c [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:26.295478 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 7694
I20260812 16:41:26.295588 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:41:26.295807 1015 external_mini_cluster.cc:1428] Setting key fa5077ebfc6a340ff093539556d7db4c
I20260812 16:41:26.295856 7822 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:41:26.296411 7822 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:41:26.298249 7822 catalog_manager.cc:1347] Generated new cluster ID: 3b3e7ca364b74c528b5d9c0061c568d8
I20260812 16:41:26.298306 7822 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:41:26.298960 7835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:26.296311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43697 (local address 127.0.0.1:35843)
0812 16:41:26.296769 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:26.296781 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:26.296825 (+ 44us) server_negotiation.cc:407] Connection header received
0812 16:41:26.297163 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:26.297172 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:26.297218 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:26.297379 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:26.298519 (+ 1140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:26.298523 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:26.298523 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:41:26.298528 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:26.298575 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:41:26.298587 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:26.298655 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:26.298728 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 16:41:26.298771 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":84,"threads_started":1}
I20260812 16:41:26.299906 1015 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:41:26.304589 7835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:26.301435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33801 (local address 127.0.0.1:35843)
0812 16:41:26.301469 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:26.301470 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:26.303395 (+ 1925us) server_negotiation.cc:407] Connection header received
0812 16:41:26.303507 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:26.303508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:26.303516 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:26.303578 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:26.304268 (+ 690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:26.304270 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:26.304270 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:41:26.304295 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:26.304318 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:41:26.304323 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:26.304377 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:26.304489 (+ 112us) server_negotiation.cc:299] Negotiation successful
0812 16:41:26.304508 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260812 16:41:26.314311 7822 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:41:26.314509 7822 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:41:26.325345 7822 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b983e73790c48be916618551f4d375c: Generated new TSK 0
I20260812 16:41:26.325585 7822 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:41:26.330250 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 7694
2026-08-12T16:41:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1609 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:41:26 dist-test-slave-w4v5 krb5kdc[7856](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:41:26 dist-test-slave-w4v5 krb5kdc[7856](info): set up 2 sockets
Aug 12 16:41:26 dist-test-slave-w4v5 krb5kdc[7856](info): commencing operation
krb5kdc: starting...
W20260812 16:41:27.656599 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.293s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:41:27 dist-test-slave-w4v5 krb5kdc[7856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:41:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:41:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 16:41:27.781883 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43033
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43033
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:41:27.867328 7880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:27.867614 7880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:27.867641 7880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:27.868749 7880 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:41:27.868786 7880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:27.868801 7880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:27.868813 7880 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:41:27.868825 7880 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:41:27.870252 7880 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43033
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7880
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:27.870445 7880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:27.870641 7880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:27.875605 7888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:27.875631 7885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:27.875625 7886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:27.875964 7880 server_base.cc:1034] running on GCE node
I20260812 16:41:27.876233 7880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:27.876541 7880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:27.877805 7880 hybrid_clock.cc:648] HybridClock initialized: now 1786552887877780 us; error 39 us; skew 500 ppm
Aug 12 16:41:27 dist-test-slave-w4v5 krb5kdc[7856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552887, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:41:27.880544 7880 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 16:41:27.881057 7880 webserver.cc:460] Webserver started at http://127.0.0.1:40281/ using document root <none> and password file <none>
I20260812 16:41:27.881322 7880 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:27.881398 7880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:27.881547 7880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:27.882587 7880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "38de722f4e5c41e78e9079758c84a96a"
format_stamp: "Formatted at 2026-08-12 16:41:27 on dist-test-slave-w4v5"
server_key: "c38573bc4ca26c5f1f74c5de8bae066c"
server_key_iv: "8e61c41ff70b7bf9bff14125d6074fc5"
server_key_version: "encryptionkey@0"
I20260812 16:41:27.882956 7880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "38de722f4e5c41e78e9079758c84a96a"
format_stamp: "Formatted at 2026-08-12 16:41:27 on dist-test-slave-w4v5"
server_key: "c38573bc4ca26c5f1f74c5de8bae066c"
server_key_iv: "8e61c41ff70b7bf9bff14125d6074fc5"
server_key_version: "encryptionkey@0"
I20260812 16:41:27.884334 7880 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:41:27.885087 7900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:27.885264 7880 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:41:27.885372 7880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "38de722f4e5c41e78e9079758c84a96a"
format_stamp: "Formatted at 2026-08-12 16:41:27 on dist-test-slave-w4v5"
server_key: "c38573bc4ca26c5f1f74c5de8bae066c"
server_key_iv: "8e61c41ff70b7bf9bff14125d6074fc5"
server_key_version: "encryptionkey@0"
I20260812 16:41:27.885448 7880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:27.901978 7880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:27.904435 7880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:27.904609 7880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:27.909399 7880 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43033
I20260812 16:41:27.909436 8001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43033 every 8 connection(s)
I20260812 16:41:27.909832 7880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:41:27.910863 8002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:27.913211 8002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38de722f4e5c41e78e9079758c84a96a: Bootstrap starting.
I20260812 16:41:27.913810 8002 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38de722f4e5c41e78e9079758c84a96a: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:27.914112 8002 log.cc:824] T 00000000000000000000000000000000 P 38de722f4e5c41e78e9079758c84a96a: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:27.914813 8002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38de722f4e5c41e78e9079758c84a96a: No bootstrap required, opened a new log
I20260812 16:41:27.916471 8002 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38de722f4e5c41e78e9079758c84a96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38de722f4e5c41e78e9079758c84a96a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43033 } }
I20260812 16:41:27.916625 8002 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38de722f4e5c41e78e9079758c84a96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:27.916651 8002 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38de722f4e5c41e78e9079758c84a96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38de722f4e5c41e78e9079758c84a96a, State: Initialized, Role: FOLLOWER
I20260812 16:41:27.916795 8002 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38de722f4e5c41e78e9079758c84a96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38de722f4e5c41e78e9079758c84a96a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43033 } }
I20260812 16:41:27.916877 8002 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38de722f4e5c41e78e9079758c84a96a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:41:27.916903 8002 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38de722f4e5c41e78e9079758c84a96a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:41:27.916931 8002 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38de722f4e5c41e78e9079758c84a96a [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:27.917479 8002 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38de722f4e5c41e78e9079758c84a96a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38de722f4e5c41e78e9079758c84a96a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43033 } }
I20260812 16:41:27.917600 8002 leader_election.cc:302] T 00000000000000000000000000000000 P 38de722f4e5c41e78e9079758c84a96a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38de722f4e5c41e78e9079758c84a96a; no voters:
I20260812 16:41:27.917785 8002 leader_election.cc:288] T 00000000000000000000000000000000 P 38de722f4e5c41e78e9079758c84a96a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:41:27.917830 8005 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38de722f4e5c41e78e9079758c84a96a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:41:27.918071 8002 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38de722f4e5c41e78e9079758c84a96a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:41:27.918097 8005 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38de722f4e5c41e78e9079758c84a96a [term 1 LEADER]: Becoming Leader. State: Replica: 38de722f4e5c41e78e9079758c84a96a, State: Running, Role: LEADER
I20260812 16:41:27.918187 8005 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38de722f4e5c41e78e9079758c84a96a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38de722f4e5c41e78e9079758c84a96a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43033 } }
I20260812 16:41:27.918434 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 7880
I20260812 16:41:27.918571 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:41:27.918774 1015 external_mini_cluster.cc:1428] Setting key e9af599666884675355eeff4a1842c46
I20260812 16:41:27.918998 8009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38de722f4e5c41e78e9079758c84a96a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38de722f4e5c41e78e9079758c84a96a. Latest consensus state: current_term: 1 leader_uuid: "38de722f4e5c41e78e9079758c84a96a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38de722f4e5c41e78e9079758c84a96a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43033 } } }
I20260812 16:41:27.919112 8009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38de722f4e5c41e78e9079758c84a96a [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:27.919531 8008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38de722f4e5c41e78e9079758c84a96a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38de722f4e5c41e78e9079758c84a96a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38de722f4e5c41e78e9079758c84a96a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43033 } } }
I20260812 16:41:27.919621 8008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38de722f4e5c41e78e9079758c84a96a [sys.catalog]: This master's current role is: LEADER
W20260812 16:41:27.920178 8039 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 38de722f4e5c41e78e9079758c84a96a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 16:41:27.920238 8039 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:41:27.920373 8032 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:41:27.920538 8032 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:41:27.921963 8032 catalog_manager.cc:1347] Generated new cluster ID: d84a80b968394d3aa64358e842cd48cf
I20260812 16:41:27.922014 8032 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:41:27.927343 8032 catalog_manager.cc:1370] Generated new certificate authority record
Aug 12 16:41:27 dist-test-slave-w4v5 krb5kdc[7856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 16:41:27.928342 8032 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:41:27.930738 8032 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38de722f4e5c41e78e9079758c84a96a: Generated new TSK 0
I20260812 16:41:27.930935 8032 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:41:27.932258 8037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:27.919568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.75:59111 (local address 127.0.0.1:43033)
0812 16:41:27.919809 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:27.919819 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:27.920100 (+ 281us) server_negotiation.cc:407] Connection header received
0812 16:41:27.920345 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:27.920354 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:27.920420 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:27.921405 (+ 985us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:27.921502 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:27.922880 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:27.924073 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:27.924711 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:27.925171 (+ 460us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:27.928339 (+ 3168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:27.928344 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:27.928344 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:27.928357 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:27.930380 (+ 2023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:27.930903 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:27.930907 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:27.930907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:27.931014 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:27.931332 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:27.931333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:27.931333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:27.931643 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:27.931794 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:27.931937 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 16:41:27.932044 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":60,"threads_started":1}
I20260812 16:41:27.933293 1015 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:41:27.941717 8037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:27.934913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.75:47903 (local address 127.0.0.1:43033)
0812 16:41:27.934957 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:27.934959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:27.935133 (+ 174us) server_negotiation.cc:407] Connection header received
0812 16:41:27.935233 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:27.935234 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:27.935245 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:27.935302 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:27.935843 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:27.936762 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:27.937477 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:27.937812 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:27.937883 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:27.939133 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:27.939134 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:27.939135 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:27.939142 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:27.940592 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:27.940946 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:27.940948 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:27.940948 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:27.941007 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:27.941225 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:27.941226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:27.941227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:27.941354 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:27.941471 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:27.941560 (+ 89us) server_negotiation.cc:299] Negotiation successful
0812 16:41:27.941625 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 16:41:27.943934 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 7880
W20260812 16:41:27.959518 7852 connection.cc:535] client connection to 127.0.0.1:43033 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43033): Connection reset by peer (error 104)
2026-08-12T16:41:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1610 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:41:28 dist-test-slave-w4v5 krb5kdc[8057](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:41:28 dist-test-slave-w4v5 krb5kdc[8057](info): set up 2 sockets
Aug 12 16:41:28 dist-test-slave-w4v5 krb5kdc[8057](info): commencing operation
krb5kdc: starting...
W20260812 16:41:29.315711 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.342s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:41:29 dist-test-slave-w4v5 krb5kdc[8057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:41:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:41:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 16:41:29.444288 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36427
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36427
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:41:29.529361 8084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:29.529574 8084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:29.529599 8084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:29.530720 8084 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:41:29.530762 8084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:29.530776 8084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:29.530789 8084 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:41:29.530802 8084 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:41:29.532271 8084 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36427
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:29.532477 8084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:29.532653 8084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:29.538110 8096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:29.538162 8093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:29.538333 8084 server_base.cc:1034] running on GCE node
W20260812 16:41:29.538218 8092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:29.538678 8084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:29.538949 8084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:29.540127 8084 hybrid_clock.cc:648] HybridClock initialized: now 1786552889540103 us; error 41 us; skew 500 ppm
Aug 12 16:41:29 dist-test-slave-w4v5 krb5kdc[8057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552889, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:41:29.542845 8084 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 16:41:29.543397 8084 webserver.cc:460] Webserver started at http://127.0.0.1:35891/ using document root <none> and password file <none>
I20260812 16:41:29.543651 8084 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:29.543726 8084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:29.543877 8084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:29.544994 8084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "127e7ca9c61b464f9ea4ddd45d198ba1"
format_stamp: "Formatted at 2026-08-12 16:41:29 on dist-test-slave-w4v5"
server_key: "4a72092bdfa87505c79db30e4f94d33d"
server_key_iv: "3ed3bb33934e83e16e44a7e4551b129e"
server_key_version: "encryptionkey@0"
I20260812 16:41:29.545387 8084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "127e7ca9c61b464f9ea4ddd45d198ba1"
format_stamp: "Formatted at 2026-08-12 16:41:29 on dist-test-slave-w4v5"
server_key: "4a72092bdfa87505c79db30e4f94d33d"
server_key_iv: "3ed3bb33934e83e16e44a7e4551b129e"
server_key_version: "encryptionkey@0"
I20260812 16:41:29.546847 8084 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:41:29.547789 8107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:29.547968 8084 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:41:29.548087 8084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "127e7ca9c61b464f9ea4ddd45d198ba1"
format_stamp: "Formatted at 2026-08-12 16:41:29 on dist-test-slave-w4v5"
server_key: "4a72092bdfa87505c79db30e4f94d33d"
server_key_iv: "3ed3bb33934e83e16e44a7e4551b129e"
server_key_version: "encryptionkey@0"
I20260812 16:41:29.548182 8084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:29.559000 8084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:29.561085 8084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:29.561269 8084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:29.565472 8084 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36427
I20260812 16:41:29.565538 8200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36427 every 8 connection(s)
I20260812 16:41:29.565912 8084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:41:29.566960 8204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:29.569341 8204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 127e7ca9c61b464f9ea4ddd45d198ba1: Bootstrap starting.
I20260812 16:41:29.570017 8204 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 127e7ca9c61b464f9ea4ddd45d198ba1: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:29.570039 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 8084
I20260812 16:41:29.570153 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:41:29.570348 1015 external_mini_cluster.cc:1428] Setting key 60582301f5825f2fedb7992465bef917
I20260812 16:41:29.570385 8204 log.cc:824] T 00000000000000000000000000000000 P 127e7ca9c61b464f9ea4ddd45d198ba1: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:29.571280 8204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 127e7ca9c61b464f9ea4ddd45d198ba1: No bootstrap required, opened a new log
I20260812 16:41:29.573770 8204 raft_consensus.cc:348] T 00000000000000000000000000000000 P 127e7ca9c61b464f9ea4ddd45d198ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "127e7ca9c61b464f9ea4ddd45d198ba1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36427 } }
I20260812 16:41:29.573943 8204 raft_consensus.cc:374] T 00000000000000000000000000000000 P 127e7ca9c61b464f9ea4ddd45d198ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:29.573974 8204 raft_consensus.cc:729] T 00000000000000000000000000000000 P 127e7ca9c61b464f9ea4ddd45d198ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 127e7ca9c61b464f9ea4ddd45d198ba1, State: Initialized, Role: FOLLOWER
I20260812 16:41:29.574090 8204 consensus_queue.cc:260] T 00000000000000000000000000000000 P 127e7ca9c61b464f9ea4ddd45d198ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "127e7ca9c61b464f9ea4ddd45d198ba1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36427 } }
I20260812 16:41:29.574170 8204 raft_consensus.cc:388] T 00000000000000000000000000000000 P 127e7ca9c61b464f9ea4ddd45d198ba1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:41:29.574203 8204 raft_consensus.cc:482] T 00000000000000000000000000000000 P 127e7ca9c61b464f9ea4ddd45d198ba1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:41:29.574249 8204 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 127e7ca9c61b464f9ea4ddd45d198ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:29.574817 8204 raft_consensus.cc:504] T 00000000000000000000000000000000 P 127e7ca9c61b464f9ea4ddd45d198ba1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "127e7ca9c61b464f9ea4ddd45d198ba1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36427 } }
I20260812 16:41:29.574946 8204 leader_election.cc:302] T 00000000000000000000000000000000 P 127e7ca9c61b464f9ea4ddd45d198ba1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 127e7ca9c61b464f9ea4ddd45d198ba1; no voters:
I20260812 16:41:29.575151 8204 leader_election.cc:288] T 00000000000000000000000000000000 P 127e7ca9c61b464f9ea4ddd45d198ba1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:41:29.575187 8217 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 127e7ca9c61b464f9ea4ddd45d198ba1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:41:29.575367 8217 raft_consensus.cc:686] T 00000000000000000000000000000000 P 127e7ca9c61b464f9ea4ddd45d198ba1 [term 1 LEADER]: Becoming Leader. State: Replica: 127e7ca9c61b464f9ea4ddd45d198ba1, State: Running, Role: LEADER
I20260812 16:41:29.575446 8204 sys_catalog.cc:564] T 00000000000000000000000000000000 P 127e7ca9c61b464f9ea4ddd45d198ba1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:41:29.575590 8217 consensus_queue.cc:237] T 00000000000000000000000000000000 P 127e7ca9c61b464f9ea4ddd45d198ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "127e7ca9c61b464f9ea4ddd45d198ba1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36427 } }
I20260812 16:41:29.576555 8214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:29.570963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.75:43717 (local address 127.0.0.1:36427)
0812 16:41:29.571215 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:29.571223 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:29.571251 (+ 28us) server_negotiation.cc:407] Connection header received
0812 16:41:29.571745 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:29.571753 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:29.571806 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:29.571928 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:29.572208 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:29.573210 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:29.574338 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:29.574820 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:29.574945 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:29.575781 (+ 836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:29.575786 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:29.575786 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:41:29.576018 (+ 232us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:29.576063 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:41:29.576092 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:29.576194 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:29.576290 (+ 96us) server_negotiation.cc:299] Negotiation successful
0812 16:41:29.576320 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":144,"threads_started":1}
I20260812 16:41:29.577237 8218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 127e7ca9c61b464f9ea4ddd45d198ba1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "127e7ca9c61b464f9ea4ddd45d198ba1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "127e7ca9c61b464f9ea4ddd45d198ba1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36427 } } }
I20260812 16:41:29.577329 8218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 127e7ca9c61b464f9ea4ddd45d198ba1 [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:29.577528 8219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 127e7ca9c61b464f9ea4ddd45d198ba1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 127e7ca9c61b464f9ea4ddd45d198ba1. Latest consensus state: current_term: 1 leader_uuid: "127e7ca9c61b464f9ea4ddd45d198ba1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "127e7ca9c61b464f9ea4ddd45d198ba1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36427 } } }
I20260812 16:41:29.577605 8219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 127e7ca9c61b464f9ea4ddd45d198ba1 [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:29.577761 8244 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:41:29.578239 1015 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:41:29.578298 8244 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:41:29.579901 8244 catalog_manager.cc:1347] Generated new cluster ID: bc3a10e4e662467e84d809ad897d5a9b
I20260812 16:41:29.579958 8244 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:41:29.586506 8214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:29.583456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.75:52527 (local address 127.0.0.1:36427)
0812 16:41:29.583494 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:29.583496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:29.583723 (+ 227us) server_negotiation.cc:407] Connection header received
0812 16:41:29.583781 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:29.583782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:29.583796 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:29.583846 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:29.584007 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:29.584492 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:29.585264 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:29.585502 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:29.585591 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:29.586276 (+ 685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:29.586277 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:29.586278 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:41:29.586282 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:29.586297 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:41:29.586301 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:29.586355 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:29.586416 (+ 61us) server_negotiation.cc:299] Negotiation successful
0812 16:41:29.586426 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 16:41:29.590387 8244 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:41:29.591176 8244 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:41:29.601238 8244 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 127e7ca9c61b464f9ea4ddd45d198ba1: Generated new TSK 0
I20260812 16:41:29.601452 8244 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:41:29.608321 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 8084
W20260812 16:41:29.628865 8055 connection.cc:535] client connection to 127.0.0.1:36427 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36427): Connection reset by peer (error 104)
2026-08-12T16:41:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1669 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:41:29 dist-test-slave-w4v5 krb5kdc[8267](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:41:29 dist-test-slave-w4v5 krb5kdc[8267](info): set up 2 sockets
Aug 12 16:41:29 dist-test-slave-w4v5 krb5kdc[8267](info): commencing operation
krb5kdc: starting...
W20260812 16:41:30.953286 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.310s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:41:31 dist-test-slave-w4v5 krb5kdc[8267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:41:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:41:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 16:41:31.082882 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35101
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35101
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:41:31.167477 8292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:31.167745 8292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:31.167789 8292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:31.168991 8292 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:41:31.169061 8292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:31.169087 8292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:31.169118 8292 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:41:31.169152 8292 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:41:31.170661 8292 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35101
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:31.170946 8292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:31.171239 8292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:31.176465 8304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:31.176453 8300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:31.176501 8299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:31.176671 8292 server_base.cc:1034] running on GCE node
I20260812 16:41:31.176971 8292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:31.177214 8292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:31.178357 8292 hybrid_clock.cc:648] HybridClock initialized: now 1786552891178343 us; error 30 us; skew 500 ppm
Aug 12 16:41:31 dist-test-slave-w4v5 krb5kdc[8267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552891, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:41:31.181005 8292 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 16:41:31.181535 8292 webserver.cc:460] Webserver started at http://127.0.0.1:43203/ using document root <none> and password file <none>
I20260812 16:41:31.181782 8292 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:31.181854 8292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:31.182005 8292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:31.183168 8292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "c904ff6359f64df0b1e09ec2dc6c3cfb"
format_stamp: "Formatted at 2026-08-12 16:41:31 on dist-test-slave-w4v5"
server_key: "d3640b82eb0d7b6c4a28cc97d18bbe08"
server_key_iv: "2dfeb6714205742ea9ffe8b7e2c0d557"
server_key_version: "encryptionkey@0"
I20260812 16:41:31.183553 8292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "c904ff6359f64df0b1e09ec2dc6c3cfb"
format_stamp: "Formatted at 2026-08-12 16:41:31 on dist-test-slave-w4v5"
server_key: "d3640b82eb0d7b6c4a28cc97d18bbe08"
server_key_iv: "2dfeb6714205742ea9ffe8b7e2c0d557"
server_key_version: "encryptionkey@0"
I20260812 16:41:31.184978 8292 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 16:41:31.185837 8324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:31.186062 8292 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:41:31.186172 8292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "c904ff6359f64df0b1e09ec2dc6c3cfb"
format_stamp: "Formatted at 2026-08-12 16:41:31 on dist-test-slave-w4v5"
server_key: "d3640b82eb0d7b6c4a28cc97d18bbe08"
server_key_iv: "2dfeb6714205742ea9ffe8b7e2c0d557"
server_key_version: "encryptionkey@0"
I20260812 16:41:31.186251 8292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:31.189832 8292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:31.192044 8292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:31.192202 8292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:31.196456 8292 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35101
I20260812 16:41:31.196489 8414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35101 every 8 connection(s)
I20260812 16:41:31.196827 8292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:41:31.197877 8417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:31.198486 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 8292
I20260812 16:41:31.198628 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:41:31.198830 1015 external_mini_cluster.cc:1428] Setting key f94e21a8c12751466002e6bdfba19422
I20260812 16:41:31.201151 8417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c904ff6359f64df0b1e09ec2dc6c3cfb: Bootstrap starting.
I20260812 16:41:31.201423 8420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:31.199430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.75:43521 (local address 127.0.0.1:35101)
0812 16:41:31.199604 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:31.199614 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:31.199654 (+ 40us) server_negotiation.cc:407] Connection header received
0812 16:41:31.200051 (+ 397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:31.200058 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:31.200082 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:31.200239 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:31.200949 (+ 710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:31.200952 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:31.200953 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:41:31.200956 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:31.200994 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:41:31.201004 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:31.201060 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:31.201155 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 16:41:31.201237 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":82,"threads_started":1}
I20260812 16:41:31.201915 8417 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c904ff6359f64df0b1e09ec2dc6c3cfb: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:31.202291 8417 log.cc:824] T 00000000000000000000000000000000 P c904ff6359f64df0b1e09ec2dc6c3cfb: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:31.202471 1015 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:41:31.203050 8417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c904ff6359f64df0b1e09ec2dc6c3cfb: No bootstrap required, opened a new log
I20260812 16:41:31.205093 8420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:31.204059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.75:57105 (local address 127.0.0.1:35101)
0812 16:41:31.204099 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:31.204101 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:31.204211 (+ 110us) server_negotiation.cc:407] Connection header received
0812 16:41:31.204284 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:31.204285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:31.204288 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:31.204334 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:31.204883 (+ 549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:31.204884 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:31.204884 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:41:31.204888 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:31.204900 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:41:31.204904 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:31.204947 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:31.205006 (+ 59us) server_negotiation.cc:299] Negotiation successful
0812 16:41:31.205017 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 16:41:31.205426 8417 raft_consensus.cc:348] T 00000000000000000000000000000000 P c904ff6359f64df0b1e09ec2dc6c3cfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c904ff6359f64df0b1e09ec2dc6c3cfb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35101 } }
I20260812 16:41:31.205611 8417 raft_consensus.cc:374] T 00000000000000000000000000000000 P c904ff6359f64df0b1e09ec2dc6c3cfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:31.205658 8417 raft_consensus.cc:729] T 00000000000000000000000000000000 P c904ff6359f64df0b1e09ec2dc6c3cfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c904ff6359f64df0b1e09ec2dc6c3cfb, State: Initialized, Role: FOLLOWER
I20260812 16:41:31.205786 8417 consensus_queue.cc:260] T 00000000000000000000000000000000 P c904ff6359f64df0b1e09ec2dc6c3cfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c904ff6359f64df0b1e09ec2dc6c3cfb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35101 } }
I20260812 16:41:31.205880 8417 raft_consensus.cc:388] T 00000000000000000000000000000000 P c904ff6359f64df0b1e09ec2dc6c3cfb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:41:31.205924 8417 raft_consensus.cc:482] T 00000000000000000000000000000000 P c904ff6359f64df0b1e09ec2dc6c3cfb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:41:31.205976 8417 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c904ff6359f64df0b1e09ec2dc6c3cfb [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:31.206504 8417 raft_consensus.cc:504] T 00000000000000000000000000000000 P c904ff6359f64df0b1e09ec2dc6c3cfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c904ff6359f64df0b1e09ec2dc6c3cfb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35101 } }
I20260812 16:41:31.206633 8417 leader_election.cc:302] T 00000000000000000000000000000000 P c904ff6359f64df0b1e09ec2dc6c3cfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c904ff6359f64df0b1e09ec2dc6c3cfb; no voters:
I20260812 16:41:31.206835 8417 leader_election.cc:288] T 00000000000000000000000000000000 P c904ff6359f64df0b1e09ec2dc6c3cfb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:41:31.206872 8431 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c904ff6359f64df0b1e09ec2dc6c3cfb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:41:31.207051 8431 raft_consensus.cc:686] T 00000000000000000000000000000000 P c904ff6359f64df0b1e09ec2dc6c3cfb [term 1 LEADER]: Becoming Leader. State: Replica: c904ff6359f64df0b1e09ec2dc6c3cfb, State: Running, Role: LEADER
I20260812 16:41:31.207119 8417 sys_catalog.cc:564] T 00000000000000000000000000000000 P c904ff6359f64df0b1e09ec2dc6c3cfb [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:41:31.207261 8431 consensus_queue.cc:237] T 00000000000000000000000000000000 P c904ff6359f64df0b1e09ec2dc6c3cfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c904ff6359f64df0b1e09ec2dc6c3cfb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35101 } }
I20260812 16:41:31.207659 8434 sys_catalog.cc:455] T 00000000000000000000000000000000 P c904ff6359f64df0b1e09ec2dc6c3cfb [sys.catalog]: SysCatalogTable state changed. Reason: New leader c904ff6359f64df0b1e09ec2dc6c3cfb. Latest consensus state: current_term: 1 leader_uuid: "c904ff6359f64df0b1e09ec2dc6c3cfb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c904ff6359f64df0b1e09ec2dc6c3cfb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35101 } } }
I20260812 16:41:31.207624 8433 sys_catalog.cc:455] T 00000000000000000000000000000000 P c904ff6359f64df0b1e09ec2dc6c3cfb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c904ff6359f64df0b1e09ec2dc6c3cfb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c904ff6359f64df0b1e09ec2dc6c3cfb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35101 } } }
I20260812 16:41:31.207793 8434 sys_catalog.cc:458] T 00000000000000000000000000000000 P c904ff6359f64df0b1e09ec2dc6c3cfb [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:31.207865 8433 sys_catalog.cc:458] T 00000000000000000000000000000000 P c904ff6359f64df0b1e09ec2dc6c3cfb [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:31.208041 8443 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:41:31.208595 8443 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:41:31.210151 8443 catalog_manager.cc:1347] Generated new cluster ID: c1aa1c8b7b924e9587f850f8b7dd31f6
I20260812 16:41:31.210192 8443 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:41:31.229312 8443 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:41:31.229527 8443 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:41:31.242856 8443 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c904ff6359f64df0b1e09ec2dc6c3cfb: Generated new TSK 0
I20260812 16:41:31.243119 8443 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:41:31.248816 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 8292
2026-08-12T16:41:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1644 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:41:31 dist-test-slave-w4v5 krb5kdc[8467](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:41:31 dist-test-slave-w4v5 krb5kdc[8467](info): set up 2 sockets
Aug 12 16:41:31 dist-test-slave-w4v5 krb5kdc[8467](info): commencing operation
krb5kdc: starting...
W20260812 16:41:32.570093 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.283s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:41:32 dist-test-slave-w4v5 krb5kdc[8467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:41:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:41:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 16:41:32.701653 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33915
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33915
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:41:32.790925 8487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:32.791246 8487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:32.791289 8487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:32.792497 8487 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:41:32.792568 8487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:32.792593 8487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:32.792624 8487 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:41:32.792651 8487 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:41:32.794330 8487 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33915
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:32.794629 8487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:32.794926 8487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:32.800006 8497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:32.800098 8498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:32.800009 8500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:32.800362 8487 server_base.cc:1034] running on GCE node
I20260812 16:41:32.800644 8487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:32.801007 8487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:32.802191 8487 hybrid_clock.cc:648] HybridClock initialized: now 1786552892802185 us; error 41 us; skew 500 ppm
Aug 12 16:41:32 dist-test-slave-w4v5 krb5kdc[8467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552892, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:41:32.805536 8487 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 16:41:32.806151 8487 webserver.cc:460] Webserver started at http://127.0.0.1:41555/ using document root <none> and password file <none>
I20260812 16:41:32.806460 8487 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:32.806533 8487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:32.806685 8487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:32.807947 8487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "6148a448dc4c4822a5a4e00e701393e5"
format_stamp: "Formatted at 2026-08-12 16:41:32 on dist-test-slave-w4v5"
server_key: "3a056b131b935740aa644897a3e05c39"
server_key_iv: "85d20e8d608d7f2bec472cd909df0a34"
server_key_version: "encryptionkey@0"
I20260812 16:41:32.808349 8487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "6148a448dc4c4822a5a4e00e701393e5"
format_stamp: "Formatted at 2026-08-12 16:41:32 on dist-test-slave-w4v5"
server_key: "3a056b131b935740aa644897a3e05c39"
server_key_iv: "85d20e8d608d7f2bec472cd909df0a34"
server_key_version: "encryptionkey@0"
I20260812 16:41:32.809813 8487 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:41:32.810636 8514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:32.810825 8487 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:41:32.810930 8487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "6148a448dc4c4822a5a4e00e701393e5"
format_stamp: "Formatted at 2026-08-12 16:41:32 on dist-test-slave-w4v5"
server_key: "3a056b131b935740aa644897a3e05c39"
server_key_iv: "85d20e8d608d7f2bec472cd909df0a34"
server_key_version: "encryptionkey@0"
I20260812 16:41:32.810987 8487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:32.851029 8487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:32.853615 8487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:32.853749 8487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:32.858299 8487 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33915
I20260812 16:41:32.858363 8601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33915 every 8 connection(s)
I20260812 16:41:32.858760 8487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:41:32.859929 8603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:32.862342 8603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6148a448dc4c4822a5a4e00e701393e5: Bootstrap starting.
I20260812 16:41:32.862989 8603 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6148a448dc4c4822a5a4e00e701393e5: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:32.863314 8603 log.cc:824] T 00000000000000000000000000000000 P 6148a448dc4c4822a5a4e00e701393e5: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:32.864090 8603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6148a448dc4c4822a5a4e00e701393e5: No bootstrap required, opened a new log
I20260812 16:41:32.865797 8603 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6148a448dc4c4822a5a4e00e701393e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6148a448dc4c4822a5a4e00e701393e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33915 } }
I20260812 16:41:32.865940 8603 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6148a448dc4c4822a5a4e00e701393e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:32.865964 8603 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6148a448dc4c4822a5a4e00e701393e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6148a448dc4c4822a5a4e00e701393e5, State: Initialized, Role: FOLLOWER
I20260812 16:41:32.866143 8603 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6148a448dc4c4822a5a4e00e701393e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6148a448dc4c4822a5a4e00e701393e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33915 } }
I20260812 16:41:32.866209 8603 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6148a448dc4c4822a5a4e00e701393e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:41:32.866271 8603 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6148a448dc4c4822a5a4e00e701393e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:41:32.866328 8603 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6148a448dc4c4822a5a4e00e701393e5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:32.866910 8603 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6148a448dc4c4822a5a4e00e701393e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6148a448dc4c4822a5a4e00e701393e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33915 } }
I20260812 16:41:32.867030 8603 leader_election.cc:302] T 00000000000000000000000000000000 P 6148a448dc4c4822a5a4e00e701393e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6148a448dc4c4822a5a4e00e701393e5; no voters:
I20260812 16:41:32.867266 8603 leader_election.cc:288] T 00000000000000000000000000000000 P 6148a448dc4c4822a5a4e00e701393e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:41:32.867291 8608 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6148a448dc4c4822a5a4e00e701393e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:41:32.867511 8608 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6148a448dc4c4822a5a4e00e701393e5 [term 1 LEADER]: Becoming Leader. State: Replica: 6148a448dc4c4822a5a4e00e701393e5, State: Running, Role: LEADER
I20260812 16:41:32.867556 8603 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6148a448dc4c4822a5a4e00e701393e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:41:32.867654 8608 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6148a448dc4c4822a5a4e00e701393e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6148a448dc4c4822a5a4e00e701393e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33915 } }
I20260812 16:41:32.868021 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 8487
I20260812 16:41:32.868146 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:41:32.868376 1015 external_mini_cluster.cc:1428] Setting key 102f413931b97d6a804e62bd89ca7613
I20260812 16:41:32.868827 8610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6148a448dc4c4822a5a4e00e701393e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6148a448dc4c4822a5a4e00e701393e5. Latest consensus state: current_term: 1 leader_uuid: "6148a448dc4c4822a5a4e00e701393e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6148a448dc4c4822a5a4e00e701393e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33915 } } }
I20260812 16:41:32.868937 8610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6148a448dc4c4822a5a4e00e701393e5 [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:32.869316 8609 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6148a448dc4c4822a5a4e00e701393e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6148a448dc4c4822a5a4e00e701393e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6148a448dc4c4822a5a4e00e701393e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33915 } } }
I20260812 16:41:32.869462 8609 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6148a448dc4c4822a5a4e00e701393e5 [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:32.869392 8635 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:41:32.870057 8635 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:41:32.871630 8635 catalog_manager.cc:1347] Generated new cluster ID: 9aa8b958f2844a10852776b56879b6b5
I20260812 16:41:32.871686 8635 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 16:41:32 dist-test-slave-w4v5 krb5kdc[8467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 16:41:32.880721 8637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:32.869055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.75:44265 (local address 127.0.0.1:33915)
0812 16:41:32.869241 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:32.869254 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:32.869291 (+ 37us) server_negotiation.cc:407] Connection header received
0812 16:41:32.869415 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:32.869423 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:32.869489 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:32.869652 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:32.870507 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:32.872129 (+ 1622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:32.873164 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:32.873825 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:32.873944 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:32.876819 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:32.876824 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:32.876824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:32.876837 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:32.878943 (+ 2106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:32.879418 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:32.879421 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:32.879422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:32.879547 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:32.879795 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:32.879796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:32.879797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:32.880119 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:32.880296 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:32.880381 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 16:41:32.880487 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":64,"threads_started":1}
I20260812 16:41:32.881770 1015 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:41:32.891547 8637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:32.883315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.75:57193 (local address 127.0.0.1:33915)
0812 16:41:32.883352 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:32.883353 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:32.883365 (+ 12us) server_negotiation.cc:407] Connection header received
0812 16:41:32.883439 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:32.883439 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:32.883449 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:32.883504 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:32.884079 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:32.884874 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:32.885827 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:32.886160 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:32.886231 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:32.887563 (+ 1332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:32.887564 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:32.887565 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:32.887572 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:32.890157 (+ 2585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:32.890595 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:32.890597 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:32.890597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:32.890657 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:32.890893 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:32.890894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:32.890894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:32.891026 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:32.891170 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:32.891280 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 16:41:32.891347 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 16:41:32.895474 8635 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:41:32.895973 8635 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:41:32.900271 8635 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6148a448dc4c4822a5a4e00e701393e5: Generated new TSK 0
I20260812 16:41:32.900475 8635 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:41:32.906104 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 8487
W20260812 16:41:32.921902 8462 connection.cc:535] client connection to 127.0.0.1:33915 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33915): Connection reset by peer (error 104)
2026-08-12T16:41:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1648 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:41:32 dist-test-slave-w4v5 krb5kdc[8653](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:41:32 dist-test-slave-w4v5 krb5kdc[8653](info): set up 2 sockets
Aug 12 16:41:32 dist-test-slave-w4v5 krb5kdc[8653](info): commencing operation
krb5kdc: starting...
W20260812 16:41:34.227630 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.291s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:41:34 dist-test-slave-w4v5 krb5kdc[8653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:41:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:41:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 16:41:34.358193 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36329
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36329
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:41:34.442762 8679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:34.443023 8679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:34.443063 8679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:34.444226 8679 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:41:34.444303 8679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:34.444326 8679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:34.444347 8679 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:41:34.444377 8679 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:41:34.445919 8679 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36329
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:34.446208 8679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:34.446442 8679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:34.451392 8692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:34.451422 8688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:34.451426 8690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:34.451725 8679 server_base.cc:1034] running on GCE node
I20260812 16:41:34.451980 8679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:34.452229 8679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:34.453401 8679 hybrid_clock.cc:648] HybridClock initialized: now 1786552894453388 us; error 31 us; skew 500 ppm
Aug 12 16:41:34 dist-test-slave-w4v5 krb5kdc[8653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552894, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:41:34.455921 8679 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 16:41:34.456383 8679 webserver.cc:460] Webserver started at http://127.0.0.1:37715/ using document root <none> and password file <none>
I20260812 16:41:34.456573 8679 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:34.456614 8679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:34.456703 8679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:34.457759 8679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "46072db9df2647638b2e943a7c3d6088"
format_stamp: "Formatted at 2026-08-12 16:41:34 on dist-test-slave-w4v5"
server_key: "56e11337693f5488faba39afe08ab2ea"
server_key_iv: "7e049efbdef9d814145637573505163d"
server_key_version: "encryptionkey@0"
I20260812 16:41:34.458063 8679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "46072db9df2647638b2e943a7c3d6088"
format_stamp: "Formatted at 2026-08-12 16:41:34 on dist-test-slave-w4v5"
server_key: "56e11337693f5488faba39afe08ab2ea"
server_key_iv: "7e049efbdef9d814145637573505163d"
server_key_version: "encryptionkey@0"
I20260812 16:41:34.459424 8679 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 16:41:34.460229 8704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:34.460417 8679 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:41:34.460505 8679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "46072db9df2647638b2e943a7c3d6088"
format_stamp: "Formatted at 2026-08-12 16:41:34 on dist-test-slave-w4v5"
server_key: "56e11337693f5488faba39afe08ab2ea"
server_key_iv: "7e049efbdef9d814145637573505163d"
server_key_version: "encryptionkey@0"
I20260812 16:41:34.460561 8679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:34.484969 8679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:34.486969 8679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:34.487116 8679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:34.491618 8679 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36329
I20260812 16:41:34.491675 8788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36329 every 8 connection(s)
I20260812 16:41:34.492051 8679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:41:34.493036 8791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:34.494237 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 8679
I20260812 16:41:34.494330 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:41:34.494531 1015 external_mini_cluster.cc:1428] Setting key 7ccb391d43157ea2d0901385caa098c0
I20260812 16:41:34.496027 8791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46072db9df2647638b2e943a7c3d6088: Bootstrap starting.
I20260812 16:41:34.496812 8791 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46072db9df2647638b2e943a7c3d6088: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:34.497138 8791 log.cc:824] T 00000000000000000000000000000000 P 46072db9df2647638b2e943a7c3d6088: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:34.497916 8791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46072db9df2647638b2e943a7c3d6088: No bootstrap required, opened a new log
I20260812 16:41:34.500341 8791 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46072db9df2647638b2e943a7c3d6088 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46072db9df2647638b2e943a7c3d6088" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36329 } }
I20260812 16:41:34.500519 8791 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46072db9df2647638b2e943a7c3d6088 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:34.500559 8791 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46072db9df2647638b2e943a7c3d6088 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46072db9df2647638b2e943a7c3d6088, State: Initialized, Role: FOLLOWER
I20260812 16:41:34.500736 8791 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46072db9df2647638b2e943a7c3d6088 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46072db9df2647638b2e943a7c3d6088" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36329 } }
I20260812 16:41:34.500830 8791 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46072db9df2647638b2e943a7c3d6088 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:41:34.500877 8791 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46072db9df2647638b2e943a7c3d6088 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:41:34.500921 8791 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46072db9df2647638b2e943a7c3d6088 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:34.500988 8795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:34.495114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.75:53177 (local address 127.0.0.1:36329)
0812 16:41:34.495311 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:34.495321 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:34.495359 (+ 38us) server_negotiation.cc:407] Connection header received
0812 16:41:34.495913 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:34.495922 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:34.495987 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:34.496149 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:34.496381 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:34.497809 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:34.498637 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:34.499289 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:34.499404 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:34.500116 (+ 712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:34.500120 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:34.500121 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:41:34.500327 (+ 206us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:34.500370 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:41:34.500387 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:34.500477 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:34.500714 (+ 237us) server_negotiation.cc:299] Negotiation successful
0812 16:41:34.500747 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":83,"threads_started":1}
I20260812 16:41:34.501591 8791 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46072db9df2647638b2e943a7c3d6088 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46072db9df2647638b2e943a7c3d6088" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36329 } }
I20260812 16:41:34.501731 8791 leader_election.cc:302] T 00000000000000000000000000000000 P 46072db9df2647638b2e943a7c3d6088 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46072db9df2647638b2e943a7c3d6088; no voters:
I20260812 16:41:34.501889 8791 leader_election.cc:288] T 00000000000000000000000000000000 P 46072db9df2647638b2e943a7c3d6088 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:41:34.501910 8799 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46072db9df2647638b2e943a7c3d6088 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:41:34.502031 8799 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46072db9df2647638b2e943a7c3d6088 [term 1 LEADER]: Becoming Leader. State: Replica: 46072db9df2647638b2e943a7c3d6088, State: Running, Role: LEADER
I20260812 16:41:34.502158 8791 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46072db9df2647638b2e943a7c3d6088 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:41:34.502279 8799 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46072db9df2647638b2e943a7c3d6088 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46072db9df2647638b2e943a7c3d6088" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36329 } }
I20260812 16:41:34.502322 1015 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:41:34.503151 8802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46072db9df2647638b2e943a7c3d6088 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46072db9df2647638b2e943a7c3d6088. Latest consensus state: current_term: 1 leader_uuid: "46072db9df2647638b2e943a7c3d6088" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46072db9df2647638b2e943a7c3d6088" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36329 } } }
I20260812 16:41:34.503250 8802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46072db9df2647638b2e943a7c3d6088 [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:34.503965 8800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46072db9df2647638b2e943a7c3d6088 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46072db9df2647638b2e943a7c3d6088" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46072db9df2647638b2e943a7c3d6088" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36329 } } }
I20260812 16:41:34.504065 8800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46072db9df2647638b2e943a7c3d6088 [sys.catalog]: This master's current role is: LEADER
W20260812 16:41:34.504501 8829 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 46072db9df2647638b2e943a7c3d6088: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 16:41:34.504550 8829 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:41:34.504622 8830 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:41:34.504797 8830 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:41:34.506533 8830 catalog_manager.cc:1347] Generated new cluster ID: 33d14793dbd14bf6b649190b72ed0bfe
I20260812 16:41:34.506594 8830 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:41:34.507897 8795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:34.504420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.75:41989 (local address 127.0.0.1:36329)
0812 16:41:34.504455 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:34.504457 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:34.504600 (+ 143us) server_negotiation.cc:407] Connection header received
0812 16:41:34.504649 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:34.504649 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:34.504661 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:34.504729 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:34.504869 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:34.505554 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:34.506529 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:34.506804 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:34.506882 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:34.507628 (+ 746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:34.507629 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:34.507630 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:41:34.507634 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:34.507649 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:41:34.507653 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:34.507730 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:34.507794 (+ 64us) server_negotiation.cc:299] Negotiation successful
0812 16:41:34.507803 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 16:41:34.512502 8830 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:41:34.513067 8830 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:41:34.521811 8830 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46072db9df2647638b2e943a7c3d6088: Generated new TSK 0
I20260812 16:41:34.522049 8830 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:41:34.527132 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 8679
W20260812 16:41:34.546190 8650 connection.cc:535] client connection to 127.0.0.1:36329 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36329): Connection reset by peer (error 104)
2026-08-12T16:41:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1623 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:41:34 dist-test-slave-w4v5 krb5kdc[8839](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:41:34 dist-test-slave-w4v5 krb5kdc[8839](info): set up 2 sockets
Aug 12 16:41:34 dist-test-slave-w4v5 krb5kdc[8839](info): commencing operation
krb5kdc: starting...
W20260812 16:41:35.904201 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.344s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:41:35 dist-test-slave-w4v5 krb5kdc[8839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:41:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:41:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 16:41:36.038484 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34453
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34453
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:41:36.123238 8859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:36.123451 8859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:36.123476 8859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:36.124614 8859 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:41:36.124658 8859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:36.124673 8859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:36.124686 8859 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:41:36.124697 8859 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:41:36.126124 8859 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34453
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552619305671-1015-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:36.126329 8859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:36.126498 8859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:36.131750 8868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:36.131767 8859 server_base.cc:1034] running on GCE node
W20260812 16:41:36.131763 8872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:36.131920 8869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:36.132282 8859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:36.132638 8859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:36.133837 8859 hybrid_clock.cc:648] HybridClock initialized: now 1786552896133830 us; error 38 us; skew 500 ppm
Aug 12 16:41:36 dist-test-slave-w4v5 krb5kdc[8839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552896, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:41:36.141364 8859 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 16:41:36.141788 8859 webserver.cc:460] Webserver started at http://127.0.0.1:44535/ using document root <none> and password file <none>
I20260812 16:41:36.142007 8859 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:36.142048 8859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:36.142140 8859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:36.143287 8859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "b64827700a0246229896a3b85fdd5191"
format_stamp: "Formatted at 2026-08-12 16:41:36 on dist-test-slave-w4v5"
server_key: "636b99e58b20c8a7c0fc37d510366605"
server_key_iv: "55198d0fc6516d32f402a7f236338451"
server_key_version: "encryptionkey@0"
I20260812 16:41:36.143603 8859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "b64827700a0246229896a3b85fdd5191"
format_stamp: "Formatted at 2026-08-12 16:41:36 on dist-test-slave-w4v5"
server_key: "636b99e58b20c8a7c0fc37d510366605"
server_key_iv: "55198d0fc6516d32f402a7f236338451"
server_key_version: "encryptionkey@0"
I20260812 16:41:36.144922 8859 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 16:41:36.145653 8882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:36.145859 8859 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:41:36.145942 8859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "b64827700a0246229896a3b85fdd5191"
format_stamp: "Formatted at 2026-08-12 16:41:36 on dist-test-slave-w4v5"
server_key: "636b99e58b20c8a7c0fc37d510366605"
server_key_iv: "55198d0fc6516d32f402a7f236338451"
server_key_version: "encryptionkey@0"
I20260812 16:41:36.145999 8859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:36.149358 8859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:36.151662 8859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:36.151772 8859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:36.155779 8859 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34453
I20260812 16:41:36.155836 8960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34453 every 8 connection(s)
I20260812 16:41:36.156149 8859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:41:36.157095 8963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:36.159492 8963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b64827700a0246229896a3b85fdd5191: Bootstrap starting.
I20260812 16:41:36.160118 8963 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b64827700a0246229896a3b85fdd5191: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:36.160418 8963 log.cc:824] T 00000000000000000000000000000000 P b64827700a0246229896a3b85fdd5191: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:36.161211 8963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b64827700a0246229896a3b85fdd5191: No bootstrap required, opened a new log
I20260812 16:41:36.162853 8963 raft_consensus.cc:348] T 00000000000000000000000000000000 P b64827700a0246229896a3b85fdd5191 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b64827700a0246229896a3b85fdd5191" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34453 } }
I20260812 16:41:36.163014 8963 raft_consensus.cc:374] T 00000000000000000000000000000000 P b64827700a0246229896a3b85fdd5191 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:36.163064 8963 raft_consensus.cc:729] T 00000000000000000000000000000000 P b64827700a0246229896a3b85fdd5191 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b64827700a0246229896a3b85fdd5191, State: Initialized, Role: FOLLOWER
I20260812 16:41:36.163270 8963 consensus_queue.cc:260] T 00000000000000000000000000000000 P b64827700a0246229896a3b85fdd5191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b64827700a0246229896a3b85fdd5191" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34453 } }
I20260812 16:41:36.163373 8963 raft_consensus.cc:388] T 00000000000000000000000000000000 P b64827700a0246229896a3b85fdd5191 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:41:36.163420 8963 raft_consensus.cc:482] T 00000000000000000000000000000000 P b64827700a0246229896a3b85fdd5191 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:41:36.163470 8963 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b64827700a0246229896a3b85fdd5191 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:36.164000 8963 raft_consensus.cc:504] T 00000000000000000000000000000000 P b64827700a0246229896a3b85fdd5191 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b64827700a0246229896a3b85fdd5191" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34453 } }
I20260812 16:41:36.164124 8963 leader_election.cc:302] T 00000000000000000000000000000000 P b64827700a0246229896a3b85fdd5191 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b64827700a0246229896a3b85fdd5191; no voters:
I20260812 16:41:36.164345 8963 leader_election.cc:288] T 00000000000000000000000000000000 P b64827700a0246229896a3b85fdd5191 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:41:36.164383 8969 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b64827700a0246229896a3b85fdd5191 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:41:36.164511 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 8859
I20260812 16:41:36.164628 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:41:36.164659 8969 raft_consensus.cc:686] T 00000000000000000000000000000000 P b64827700a0246229896a3b85fdd5191 [term 1 LEADER]: Becoming Leader. State: Replica: b64827700a0246229896a3b85fdd5191, State: Running, Role: LEADER
I20260812 16:41:36.164644 8963 sys_catalog.cc:564] T 00000000000000000000000000000000 P b64827700a0246229896a3b85fdd5191 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:41:36.164826 1015 external_mini_cluster.cc:1428] Setting key 4941b3cfa10ae28dead61dff3a1c4c2f
I20260812 16:41:36.165479 8969 consensus_queue.cc:237] T 00000000000000000000000000000000 P b64827700a0246229896a3b85fdd5191 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b64827700a0246229896a3b85fdd5191" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34453 } }
I20260812 16:41:36.166164 8972 sys_catalog.cc:455] T 00000000000000000000000000000000 P b64827700a0246229896a3b85fdd5191 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b64827700a0246229896a3b85fdd5191. Latest consensus state: current_term: 1 leader_uuid: "b64827700a0246229896a3b85fdd5191" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b64827700a0246229896a3b85fdd5191" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34453 } } }
I20260812 16:41:36.166250 8972 sys_catalog.cc:458] T 00000000000000000000000000000000 P b64827700a0246229896a3b85fdd5191 [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:36.166230 8970 sys_catalog.cc:455] T 00000000000000000000000000000000 P b64827700a0246229896a3b85fdd5191 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b64827700a0246229896a3b85fdd5191" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b64827700a0246229896a3b85fdd5191" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34453 } } }
I20260812 16:41:36.166321 8970 sys_catalog.cc:458] T 00000000000000000000000000000000 P b64827700a0246229896a3b85fdd5191 [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:36.166651 8993 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:41:36.167275 8993 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:41:36.168007 8982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:36.165365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.75:59613 (local address 127.0.0.1:34453)
0812 16:41:36.165799 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:36.165807 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:36.165859 (+ 52us) server_negotiation.cc:407] Connection header received
0812 16:41:36.166522 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:36.166531 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:36.166651 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:36.166814 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:36.167554 (+ 740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:36.167558 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:36.167558 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:41:36.167563 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:36.167607 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:41:36.167622 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:36.167700 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:36.167773 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 16:41:36.167819 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":107,"threads_started":1}
I20260812 16:41:36.168988 8993 catalog_manager.cc:1347] Generated new cluster ID: 5d80c55dcb9d4bbe87ff675b49400479
I20260812 16:41:36.169018 1015 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:41:36.169040 8993 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:41:36.171756 8982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:36.170634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.75:45529 (local address 127.0.0.1:34453)
0812 16:41:36.170711 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:36.170713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:36.170770 (+ 57us) server_negotiation.cc:407] Connection header received
0812 16:41:36.170843 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:36.170844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:36.170848 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:36.170898 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:36.171515 (+ 617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:36.171517 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:36.171517 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:41:36.171521 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:36.171536 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:41:36.171540 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:36.171594 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:36.171686 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 16:41:36.171698 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 16:41:36.193413 8993 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:41:36.193629 8993 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:41:36.212529 8993 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b64827700a0246229896a3b85fdd5191: Generated new TSK 0
I20260812 16:41:36.212800 8993 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:41:36.226845 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 8859
2026-08-12T16:41:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1702 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:41:36 dist-test-slave-w4v5 krb5kdc[9008](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:41:36 dist-test-slave-w4v5 krb5kdc[9008](info): set up 2 sockets
Aug 12 16:41:36 dist-test-slave-w4v5 krb5kdc[9008](info): commencing operation
krb5kdc: starting...
W20260812 16:41:37.547842 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.285s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:41:37 dist-test-slave-w4v5 krb5kdc[9008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:41:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:41:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
I20260812 16:41:37.673381 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:42849
--webserver_interface=127.0.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:39041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:42849
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:41:37.757269 9025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:37.757483 9025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:37.757506 9025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:37.758630 9025 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:41:37.758672 9025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:37.758687 9025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:37.758699 9025 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:41:37.758711 9025 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:41:37.760241 9025 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:39041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:42849
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:42849
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:37.760443 9025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:37.760618 9025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:37.765650 9037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:37.765689 9033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:37.765687 9031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:37.765858 9025 server_base.cc:1034] running on GCE node
I20260812 16:41:37.766103 9025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:37.766368 9025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:37.767544 9025 hybrid_clock.cc:648] HybridClock initialized: now 1786552897767514 us; error 42 us; skew 500 ppm
Aug 12 16:41:37 dist-test-slave-w4v5 krb5kdc[9008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552897, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:41:37.770195 9025 init.cc:377] Logged in from keytab as kudu/127.0.253.254@KRBTEST.COM (short username kudu)
I20260812 16:41:37.770723 9025 webserver.cc:460] Webserver started at http://127.0.253.254:39913/ using document root <none> and password file <none>
I20260812 16:41:37.770960 9025 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:37.771039 9025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:37.771232 9025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:37.772320 9025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "77087bd329cc434d9f5b332ab1055c14"
format_stamp: "Formatted at 2026-08-12 16:41:37 on dist-test-slave-w4v5"
server_key: "a6ac99c60a938b9746ffc4c886ffc7c3"
server_key_iv: "37047f5aac3cff10b3b31cefa2c707a1"
server_key_version: "encryptionkey@0"
I20260812 16:41:37.772703 9025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "77087bd329cc434d9f5b332ab1055c14"
format_stamp: "Formatted at 2026-08-12 16:41:37 on dist-test-slave-w4v5"
server_key: "a6ac99c60a938b9746ffc4c886ffc7c3"
server_key_iv: "37047f5aac3cff10b3b31cefa2c707a1"
server_key_version: "encryptionkey@0"
I20260812 16:41:37.774089 9025 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:41:37.774858 9050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:37.775132 9025 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:41:37.775254 9025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "77087bd329cc434d9f5b332ab1055c14"
format_stamp: "Formatted at 2026-08-12 16:41:37 on dist-test-slave-w4v5"
server_key: "a6ac99c60a938b9746ffc4c886ffc7c3"
server_key_iv: "37047f5aac3cff10b3b31cefa2c707a1"
server_key_version: "encryptionkey@0"
I20260812 16:41:37.775342 9025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:37.790618 9025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:37.792999 9025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:37.793171 9025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:37.797514 9025 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:42849
I20260812 16:41:37.797560 9143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:42849 every 8 connection(s)
I20260812 16:41:37.797937 9025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:41:37.798520 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 9025
I20260812 16:41:37.798655 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:41:37.798858 1015 external_mini_cluster.cc:1428] Setting key 8c86b3ec20b9a1bd6cd5eee2acd5ede9
I20260812 16:41:37.798951 9144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:37.802452 9144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77087bd329cc434d9f5b332ab1055c14: Bootstrap starting.
I20260812 16:41:37.803145 9144 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77087bd329cc434d9f5b332ab1055c14: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:37.803514 9144 log.cc:824] T 00000000000000000000000000000000 P 77087bd329cc434d9f5b332ab1055c14: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:37.804286 9144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77087bd329cc434d9f5b332ab1055c14: No bootstrap required, opened a new log
Aug 12 16:41:37 dist-test-slave-w4v5 krb5kdc[9008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:41:37.806409 9144 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77087bd329cc434d9f5b332ab1055c14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77087bd329cc434d9f5b332ab1055c14" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 42849 } }
I20260812 16:41:37.806564 9144 raft_consensus.cc:374] T 00000000000000000000000000000000 P 77087bd329cc434d9f5b332ab1055c14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:37.806589 9144 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77087bd329cc434d9f5b332ab1055c14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77087bd329cc434d9f5b332ab1055c14, State: Initialized, Role: FOLLOWER
I20260812 16:41:37.806740 9144 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77087bd329cc434d9f5b332ab1055c14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77087bd329cc434d9f5b332ab1055c14" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 42849 } }
I20260812 16:41:37.806810 9144 raft_consensus.cc:388] T 00000000000000000000000000000000 P 77087bd329cc434d9f5b332ab1055c14 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:41:37.806908 9144 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77087bd329cc434d9f5b332ab1055c14 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:41:37.806969 9144 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77087bd329cc434d9f5b332ab1055c14 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:37.807567 9144 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77087bd329cc434d9f5b332ab1055c14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77087bd329cc434d9f5b332ab1055c14" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 42849 } }
I20260812 16:41:37.807662 9144 leader_election.cc:302] T 00000000000000000000000000000000 P 77087bd329cc434d9f5b332ab1055c14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 77087bd329cc434d9f5b332ab1055c14; no voters:
I20260812 16:41:37.807861 9144 leader_election.cc:288] T 00000000000000000000000000000000 P 77087bd329cc434d9f5b332ab1055c14 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:41:37.807904 9151 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77087bd329cc434d9f5b332ab1055c14 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:41:37.808053 9151 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77087bd329cc434d9f5b332ab1055c14 [term 1 LEADER]: Becoming Leader. State: Replica: 77087bd329cc434d9f5b332ab1055c14, State: Running, Role: LEADER
I20260812 16:41:37.808192 9144 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77087bd329cc434d9f5b332ab1055c14 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:41:37.808228 9151 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77087bd329cc434d9f5b332ab1055c14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77087bd329cc434d9f5b332ab1055c14" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 42849 } }
I20260812 16:41:37.808589 9154 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77087bd329cc434d9f5b332ab1055c14 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77087bd329cc434d9f5b332ab1055c14. Latest consensus state: current_term: 1 leader_uuid: "77087bd329cc434d9f5b332ab1055c14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77087bd329cc434d9f5b332ab1055c14" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 42849 } } }
I20260812 16:41:37.808677 9154 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77087bd329cc434d9f5b332ab1055c14 [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:37.809264 9156 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77087bd329cc434d9f5b332ab1055c14 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "77087bd329cc434d9f5b332ab1055c14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77087bd329cc434d9f5b332ab1055c14" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 42849 } } }
I20260812 16:41:37.809417 9156 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77087bd329cc434d9f5b332ab1055c14 [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:37.809588 9161 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:41:37.810117 9161 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:41:37.811232 9149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:37.799408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48581 (local address 127.0.253.254:42849)
0812 16:41:37.799644 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:37.799655 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:37.799691 (+ 36us) server_negotiation.cc:407] Connection header received
0812 16:41:37.800241 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:37.800250 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:37.800316 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:37.800474 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:37.801141 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:37.802535 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:37.803432 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:37.804004 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:37.804117 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:37.806930 (+ 2813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:37.806935 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:37.806935 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:37.806968 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:37.809114 (+ 2146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:37.809666 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:37.809670 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:37.809670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:37.809788 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:37.810023 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:37.810024 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:37.810024 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:37.810347 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:37.810540 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:37.810873 (+ 333us) server_negotiation.cc:299] Negotiation successful
0812 16:41:37.810984 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":101,"threads_started":1}
I20260812 16:41:37.811908 9161 catalog_manager.cc:1347] Generated new cluster ID: 30a2cf1fdebd4ccdbad056acbeedd550
I20260812 16:41:37.812166 9161 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:41:37.812282 1015 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:41:37.819880 9161 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:41:37.820395 9161 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:41:37.821378 9149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:37.813832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42991 (local address 127.0.253.254:42849)
0812 16:41:37.813868 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:37.813869 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:37.814030 (+ 161us) server_negotiation.cc:407] Connection header received
0812 16:41:37.814077 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:37.814078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:37.814088 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:37.814153 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:37.814741 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:37.815493 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:37.816477 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:37.816818 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:37.816893 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:37.818237 (+ 1344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:37.818238 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:37.818239 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:37.818246 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:37.819971 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:37.820434 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:37.820436 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:37.820436 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:37.820495 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:37.820749 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:37.820749 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:37.820750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:37.820880 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:37.820963 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:37.821216 (+ 253us) server_negotiation.cc:299] Negotiation successful
0812 16:41:37.821284 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 16:41:37.828838 9161 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 77087bd329cc434d9f5b332ab1055c14: Generated new TSK 0
I20260812 16:41:37.829059 9161 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:41:37.831948 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 9025
W20260812 16:41:37.846321 9006 connection.cc:535] client connection to 127.0.253.254:42849 recv error: Network error: failed to read from TLS socket (remote: 127.0.253.254:42849): Connection reset by peer (error 104)
2026-08-12T16:41:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1597 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:41:37 dist-test-slave-w4v5 krb5kdc[9192](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:41:37 dist-test-slave-w4v5 krb5kdc[9192](info): set up 2 sockets
Aug 12 16:41:37 dist-test-slave-w4v5 krb5kdc[9192](info): commencing operation
krb5kdc: starting...
W20260812 16:41:39.167917 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.308s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:41:39 dist-test-slave-w4v5 krb5kdc[9192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:41:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:41:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
I20260812 16:41:39.295018 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:36799
--webserver_interface=127.0.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:36343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:36799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:41:39.381104 9213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:39.381311 9213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:39.381335 9213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:39.382486 9213 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:41:39.382531 9213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:39.382546 9213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:39.382560 9213 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:41:39.382571 9213 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:41:39.384117 9213 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:36343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:36799
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:36799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:39.384382 9213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:39.384620 9213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:39.390069 9225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:39.390065 9229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:39.390190 9213 server_base.cc:1034] running on GCE node
W20260812 16:41:39.390070 9224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:39.390806 9213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:39.391120 9213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:39.392297 9213 hybrid_clock.cc:648] HybridClock initialized: now 1786552899392273 us; error 40 us; skew 500 ppm
Aug 12 16:41:39 dist-test-slave-w4v5 krb5kdc[9192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552899, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:41:39.395222 9213 init.cc:377] Logged in from keytab as kudu/127.0.253.254@KRBTEST.COM (short username kudu)
I20260812 16:41:39.395715 9213 webserver.cc:460] Webserver started at http://127.0.253.254:34429/ using document root <none> and password file <none>
I20260812 16:41:39.395990 9213 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:39.396044 9213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:39.396229 9213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:39.397466 9213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "170ee80af1fa4146865b0085a1799a34"
format_stamp: "Formatted at 2026-08-12 16:41:39 on dist-test-slave-w4v5"
server_key: "4607146cd49844a9a9d6858dfa6423e3"
server_key_iv: "28b9e72fe570a129fc746b0663fd0e65"
server_key_version: "encryptionkey@0"
I20260812 16:41:39.397955 9213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "170ee80af1fa4146865b0085a1799a34"
format_stamp: "Formatted at 2026-08-12 16:41:39 on dist-test-slave-w4v5"
server_key: "4607146cd49844a9a9d6858dfa6423e3"
server_key_iv: "28b9e72fe570a129fc746b0663fd0e65"
server_key_version: "encryptionkey@0"
I20260812 16:41:39.399544 9213 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260812 16:41:39.400441 9245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:39.400702 9213 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:41:39.400800 9213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "170ee80af1fa4146865b0085a1799a34"
format_stamp: "Formatted at 2026-08-12 16:41:39 on dist-test-slave-w4v5"
server_key: "4607146cd49844a9a9d6858dfa6423e3"
server_key_iv: "28b9e72fe570a129fc746b0663fd0e65"
server_key_version: "encryptionkey@0"
I20260812 16:41:39.400882 9213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:39.410615 9213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:39.412703 9213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:39.412868 9213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:39.417106 9213 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:36799
I20260812 16:41:39.417148 9347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:36799 every 8 connection(s)
I20260812 16:41:39.417529 9213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:41:39.418426 9350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:39.420905 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 9213
I20260812 16:41:39.420938 9350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 170ee80af1fa4146865b0085a1799a34: Bootstrap starting.
I20260812 16:41:39.421023 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:41:39.421209 1015 external_mini_cluster.cc:1428] Setting key 6c2d3e46feb26e8383fcafa7d04e09c9
I20260812 16:41:39.421730 9350 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 170ee80af1fa4146865b0085a1799a34: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:39.422063 9350 log.cc:824] T 00000000000000000000000000000000 P 170ee80af1fa4146865b0085a1799a34: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:39.422904 9350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 170ee80af1fa4146865b0085a1799a34: No bootstrap required, opened a new log
I20260812 16:41:39.425122 9350 raft_consensus.cc:348] T 00000000000000000000000000000000 P 170ee80af1fa4146865b0085a1799a34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "170ee80af1fa4146865b0085a1799a34" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36799 } }
I20260812 16:41:39.425282 9350 raft_consensus.cc:374] T 00000000000000000000000000000000 P 170ee80af1fa4146865b0085a1799a34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:39.425305 9350 raft_consensus.cc:729] T 00000000000000000000000000000000 P 170ee80af1fa4146865b0085a1799a34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170ee80af1fa4146865b0085a1799a34, State: Initialized, Role: FOLLOWER
I20260812 16:41:39.425450 9350 consensus_queue.cc:260] T 00000000000000000000000000000000 P 170ee80af1fa4146865b0085a1799a34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "170ee80af1fa4146865b0085a1799a34" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36799 } }
I20260812 16:41:39.425529 9350 raft_consensus.cc:388] T 00000000000000000000000000000000 P 170ee80af1fa4146865b0085a1799a34 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:41:39.425570 9350 raft_consensus.cc:482] T 00000000000000000000000000000000 P 170ee80af1fa4146865b0085a1799a34 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:41:39.425621 9350 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 170ee80af1fa4146865b0085a1799a34 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:39.426469 9350 raft_consensus.cc:504] T 00000000000000000000000000000000 P 170ee80af1fa4146865b0085a1799a34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "170ee80af1fa4146865b0085a1799a34" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36799 } }
I20260812 16:41:39.426573 9350 leader_election.cc:302] T 00000000000000000000000000000000 P 170ee80af1fa4146865b0085a1799a34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 170ee80af1fa4146865b0085a1799a34; no voters:
I20260812 16:41:39.426756 9350 leader_election.cc:288] T 00000000000000000000000000000000 P 170ee80af1fa4146865b0085a1799a34 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:41:39.426777 9360 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 170ee80af1fa4146865b0085a1799a34 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:41:39.426955 9360 raft_consensus.cc:686] T 00000000000000000000000000000000 P 170ee80af1fa4146865b0085a1799a34 [term 1 LEADER]: Becoming Leader. State: Replica: 170ee80af1fa4146865b0085a1799a34, State: Running, Role: LEADER
I20260812 16:41:39.427039 9360 consensus_queue.cc:237] T 00000000000000000000000000000000 P 170ee80af1fa4146865b0085a1799a34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "170ee80af1fa4146865b0085a1799a34" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36799 } }
I20260812 16:41:39.427116 9350 sys_catalog.cc:564] T 00000000000000000000000000000000 P 170ee80af1fa4146865b0085a1799a34 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:41:39.427467 9356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:39.421764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39973 (local address 127.0.253.254:36799)
0812 16:41:39.421939 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:39.421947 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:39.421973 (+ 26us) server_negotiation.cc:407] Connection header received
0812 16:41:39.422476 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:39.422483 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:39.422532 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:39.422665 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:39.422919 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:39.423903 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:39.425081 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:39.425623 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:39.425731 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:39.426435 (+ 704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:39.426439 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:39.426439 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:41:39.426657 (+ 218us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:39.426702 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:41:39.426717 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:39.426824 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:39.426996 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 16:41:39.427029 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":81,"threads_started":1}
I20260812 16:41:39.427629 9360 sys_catalog.cc:455] T 00000000000000000000000000000000 P 170ee80af1fa4146865b0085a1799a34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 170ee80af1fa4146865b0085a1799a34. Latest consensus state: current_term: 1 leader_uuid: "170ee80af1fa4146865b0085a1799a34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "170ee80af1fa4146865b0085a1799a34" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36799 } } }
I20260812 16:41:39.427707 9360 sys_catalog.cc:458] T 00000000000000000000000000000000 P 170ee80af1fa4146865b0085a1799a34 [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:39.427704 9361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 170ee80af1fa4146865b0085a1799a34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "170ee80af1fa4146865b0085a1799a34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "170ee80af1fa4146865b0085a1799a34" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36799 } } }
I20260812 16:41:39.427915 9361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 170ee80af1fa4146865b0085a1799a34 [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:39.428499 1015 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260812 16:41:39.429049 9396 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 170ee80af1fa4146865b0085a1799a34: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 16:41:39.429431 9396 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:41:39.429507 9369 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:41:39.429700 9369 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:41:39.431758 9369 catalog_manager.cc:1347] Generated new cluster ID: 2d2314a971164a00a2b24e941d0da545
I20260812 16:41:39.431814 9369 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:41:39.433370 9356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:39.430113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59781 (local address 127.0.253.254:36799)
0812 16:41:39.430148 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:39.430150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:39.430274 (+ 124us) server_negotiation.cc:407] Connection header received
0812 16:41:39.430367 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:39.430367 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:39.430377 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:39.430406 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:39.430631 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:39.431214 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:39.432099 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:39.432330 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:39.432415 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:39.433106 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:39.433108 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:39.433108 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:41:39.433113 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:39.433127 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:41:39.433131 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:39.433195 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:39.433269 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 16:41:39.433278 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 16:41:39.448107 9369 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:41:39.448829 9369 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:41:39.458302 9369 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 170ee80af1fa4146865b0085a1799a34: Generated new TSK 0
I20260812 16:41:39.458526 9369 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:41:39.465693 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 9213
W20260812 16:41:39.488996 9189 connection.cc:535] client connection to 127.0.253.254:36799 recv error: Network error: failed to read from TLS socket (remote: 127.0.253.254:36799): Connection reset by peer (error 104)
2026-08-12T16:41:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1642 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:41:39 dist-test-slave-w4v5 krb5kdc[9423](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:41:39 dist-test-slave-w4v5 krb5kdc[9423](info): set up 2 sockets
Aug 12 16:41:39 dist-test-slave-w4v5 krb5kdc[9423](info): commencing operation
krb5kdc: starting...
W20260812 16:41:40.804976 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.302s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:41:40 dist-test-slave-w4v5 krb5kdc[9423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552900, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:41:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:41:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
I20260812 16:41:40.940204 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:42531
--webserver_interface=127.0.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:34021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:42531
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:41:41.025909 9448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:41.026175 9448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:41.026216 9448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:41.027460 9448 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:41:41.027544 9448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:41.027570 9448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:41.027590 9448 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:41:41.027621 9448 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:41:41.029177 9448 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:34021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:42531
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:42531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:41.029481 9448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:41.029734 9448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:41.034994 9456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:41.035018 9462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:41.034994 9457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:41.035382 9448 server_base.cc:1034] running on GCE node
I20260812 16:41:41.035766 9448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:41.036093 9448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:41.037279 9448 hybrid_clock.cc:648] HybridClock initialized: now 1786552901037264 us; error 34 us; skew 500 ppm
Aug 12 16:41:41 dist-test-slave-w4v5 krb5kdc[9423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552901, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:41:41.040102 9448 init.cc:377] Logged in from keytab as kudu/127.0.253.254@KRBTEST.COM (short username kudu)
I20260812 16:41:41.040589 9448 webserver.cc:460] Webserver started at http://127.0.253.254:43247/ using document root <none> and password file <none>
I20260812 16:41:41.040864 9448 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:41.040938 9448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:41.041100 9448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:41.042240 9448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "8774f131a07a4c119d84c83fa2cb1259"
format_stamp: "Formatted at 2026-08-12 16:41:41 on dist-test-slave-w4v5"
server_key: "dc81b10bfdcf57adc537cc80b7629a53"
server_key_iv: "d1f3d4e2efd0d772c49ccf2f24d55fcf"
server_key_version: "encryptionkey@0"
I20260812 16:41:41.042598 9448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "8774f131a07a4c119d84c83fa2cb1259"
format_stamp: "Formatted at 2026-08-12 16:41:41 on dist-test-slave-w4v5"
server_key: "dc81b10bfdcf57adc537cc80b7629a53"
server_key_iv: "d1f3d4e2efd0d772c49ccf2f24d55fcf"
server_key_version: "encryptionkey@0"
I20260812 16:41:41.044049 9448 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:41:41.044865 9470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:41.045069 9448 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:41:41.045171 9448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "8774f131a07a4c119d84c83fa2cb1259"
format_stamp: "Formatted at 2026-08-12 16:41:41 on dist-test-slave-w4v5"
server_key: "dc81b10bfdcf57adc537cc80b7629a53"
server_key_iv: "d1f3d4e2efd0d772c49ccf2f24d55fcf"
server_key_version: "encryptionkey@0"
I20260812 16:41:41.045274 9448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:41.048756 9448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:41.050861 9448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:41.051039 9448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:41.055166 9448 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:42531
I20260812 16:41:41.055189 9573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:42531 every 8 connection(s)
I20260812 16:41:41.055552 9448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:41:41.055828 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 9448
I20260812 16:41:41.055950 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:41:41.056162 1015 external_mini_cluster.cc:1428] Setting key f6ab9b21d7e57d87ef1de6aa9d48b079
I20260812 16:41:41.056879 9574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:41.058660 9579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:41.056769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53513 (local address 127.0.253.254:42531)
0812 16:41:41.056960 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:41.056968 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:41.056993 (+ 25us) server_negotiation.cc:407] Connection header received
0812 16:41:41.057435 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:41.057442 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:41.057468 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:41.057587 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:41.058270 (+ 683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:41.058274 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:41.058275 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:41:41.058279 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:41.058317 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:41:41.058327 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:41.058385 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:41.058476 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 16:41:41.058510 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":97,"threads_started":1}
I20260812 16:41:41.059729 1015 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:41:41.060256 9574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8774f131a07a4c119d84c83fa2cb1259: Bootstrap starting.
I20260812 16:41:41.061005 9574 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8774f131a07a4c119d84c83fa2cb1259: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:41.061388 9574 log.cc:824] T 00000000000000000000000000000000 P 8774f131a07a4c119d84c83fa2cb1259: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:41.062288 9574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8774f131a07a4c119d84c83fa2cb1259: No bootstrap required, opened a new log
I20260812 16:41:41.062441 9579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:41.061379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50201 (local address 127.0.253.254:42531)
0812 16:41:41.061437 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:41.061439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:41.061447 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:41:41.061469 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:41.061470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:41.061476 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:41.061534 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:41.062206 (+ 672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:41.062207 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:41.062207 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:41:41.062211 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:41.062224 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:41:41.062228 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:41.062269 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:41.062360 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 16:41:41.062373 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 16:41:41.064548 9574 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8774f131a07a4c119d84c83fa2cb1259 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8774f131a07a4c119d84c83fa2cb1259" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 42531 } }
I20260812 16:41:41.064695 9574 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8774f131a07a4c119d84c83fa2cb1259 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:41.064720 9574 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8774f131a07a4c119d84c83fa2cb1259 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8774f131a07a4c119d84c83fa2cb1259, State: Initialized, Role: FOLLOWER
I20260812 16:41:41.064867 9574 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8774f131a07a4c119d84c83fa2cb1259 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8774f131a07a4c119d84c83fa2cb1259" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 42531 } }
I20260812 16:41:41.064939 9574 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8774f131a07a4c119d84c83fa2cb1259 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:41:41.064981 9574 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8774f131a07a4c119d84c83fa2cb1259 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:41:41.065032 9574 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8774f131a07a4c119d84c83fa2cb1259 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:41.065590 9574 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8774f131a07a4c119d84c83fa2cb1259 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8774f131a07a4c119d84c83fa2cb1259" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 42531 } }
I20260812 16:41:41.065716 9574 leader_election.cc:302] T 00000000000000000000000000000000 P 8774f131a07a4c119d84c83fa2cb1259 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8774f131a07a4c119d84c83fa2cb1259; no voters:
I20260812 16:41:41.065908 9574 leader_election.cc:288] T 00000000000000000000000000000000 P 8774f131a07a4c119d84c83fa2cb1259 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:41:41.065951 9594 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8774f131a07a4c119d84c83fa2cb1259 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:41:41.066160 9594 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8774f131a07a4c119d84c83fa2cb1259 [term 1 LEADER]: Becoming Leader. State: Replica: 8774f131a07a4c119d84c83fa2cb1259, State: Running, Role: LEADER
I20260812 16:41:41.066206 9574 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8774f131a07a4c119d84c83fa2cb1259 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:41:41.066282 9594 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8774f131a07a4c119d84c83fa2cb1259 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8774f131a07a4c119d84c83fa2cb1259" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 42531 } }
I20260812 16:41:41.066661 9597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8774f131a07a4c119d84c83fa2cb1259 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8774f131a07a4c119d84c83fa2cb1259. Latest consensus state: current_term: 1 leader_uuid: "8774f131a07a4c119d84c83fa2cb1259" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8774f131a07a4c119d84c83fa2cb1259" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 42531 } } }
I20260812 16:41:41.066656 9596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8774f131a07a4c119d84c83fa2cb1259 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8774f131a07a4c119d84c83fa2cb1259" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8774f131a07a4c119d84c83fa2cb1259" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 42531 } } }
I20260812 16:41:41.066787 9597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8774f131a07a4c119d84c83fa2cb1259 [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:41.066792 9596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8774f131a07a4c119d84c83fa2cb1259 [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:41.066993 9608 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:41:41.067515 9608 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:41:41.069130 9608 catalog_manager.cc:1347] Generated new cluster ID: 51483aaa34b2435595601e517f9a890b
I20260812 16:41:41.069190 9608 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:41:41.080494 9608 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:41:41.080685 9608 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:41:41.084308 9608 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8774f131a07a4c119d84c83fa2cb1259: Generated new TSK 0
I20260812 16:41:41.084510 9608 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:41:41.092193 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 9448
2026-08-12T16:41:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1622 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:41:41 dist-test-slave-w4v5 krb5kdc[9637](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:41:41 dist-test-slave-w4v5 krb5kdc[9637](info): set up 2 sockets
Aug 12 16:41:41 dist-test-slave-w4v5 krb5kdc[9637](info): commencing operation
krb5kdc: starting...
W20260812 16:41:42.415349 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.290s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:41:42 dist-test-slave-w4v5 krb5kdc[9637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:41:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:41:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
I20260812 16:41:42.542204 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:38763
--webserver_interface=127.0.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:34993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:38763
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:41:42.626184 9658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:42.626461 9658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:42.626503 9658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:42.627740 9658 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:41:42.627823 9658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:42.627849 9658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:42.627871 9658 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:41:42.627900 9658 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:41:42.629473 9658 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:34993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:38763
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:38763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:42.629784 9658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:42.630023 9658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:42.634964 9667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:42.635023 9665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:42.635262 9658 server_base.cc:1034] running on GCE node
W20260812 16:41:42.634982 9664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:42.635567 9658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:42.635826 9658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:42.636998 9658 hybrid_clock.cc:648] HybridClock initialized: now 1786552902636987 us; error 41 us; skew 500 ppm
Aug 12 16:41:42 dist-test-slave-w4v5 krb5kdc[9637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552902, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:41:42.639726 9658 init.cc:377] Logged in from keytab as kudu/127.0.253.254@KRBTEST.COM (short username kudu)
I20260812 16:41:42.640196 9658 webserver.cc:460] Webserver started at http://127.0.253.254:34887/ using document root <none> and password file <none>
I20260812 16:41:42.640444 9658 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:42.640516 9658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:42.640671 9658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:42.641765 9658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "1f6115e1ca6b465eabb09830e8389037"
format_stamp: "Formatted at 2026-08-12 16:41:42 on dist-test-slave-w4v5"
server_key: "171c60faa54c1dcda76d0301a5ad51d5"
server_key_iv: "043d8a758b4f80f6d96033fd9ddafb35"
server_key_version: "encryptionkey@0"
I20260812 16:41:42.642155 9658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "1f6115e1ca6b465eabb09830e8389037"
format_stamp: "Formatted at 2026-08-12 16:41:42 on dist-test-slave-w4v5"
server_key: "171c60faa54c1dcda76d0301a5ad51d5"
server_key_iv: "043d8a758b4f80f6d96033fd9ddafb35"
server_key_version: "encryptionkey@0"
I20260812 16:41:42.643610 9658 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:41:42.644403 9678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:42.644593 9658 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:41:42.644708 9658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "1f6115e1ca6b465eabb09830e8389037"
format_stamp: "Formatted at 2026-08-12 16:41:42 on dist-test-slave-w4v5"
server_key: "171c60faa54c1dcda76d0301a5ad51d5"
server_key_iv: "043d8a758b4f80f6d96033fd9ddafb35"
server_key_version: "encryptionkey@0"
I20260812 16:41:42.644788 9658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:42.663177 9658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:42.665663 9658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:42.665846 9658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:42.670171 9658 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:38763
I20260812 16:41:42.670213 9760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:38763 every 8 connection(s)
I20260812 16:41:42.670624 9658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:41:42.671695 9761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:42.674089 9761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f6115e1ca6b465eabb09830e8389037: Bootstrap starting.
I20260812 16:41:42.674691 9761 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f6115e1ca6b465eabb09830e8389037: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:42.674994 9761 log.cc:824] T 00000000000000000000000000000000 P 1f6115e1ca6b465eabb09830e8389037: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:42.675763 9761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f6115e1ca6b465eabb09830e8389037: No bootstrap required, opened a new log
I20260812 16:41:42.677330 9761 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f6115e1ca6b465eabb09830e8389037 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f6115e1ca6b465eabb09830e8389037" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 38763 } }
I20260812 16:41:42.677466 9761 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f6115e1ca6b465eabb09830e8389037 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:42.677489 9761 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f6115e1ca6b465eabb09830e8389037 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f6115e1ca6b465eabb09830e8389037, State: Initialized, Role: FOLLOWER
I20260812 16:41:42.677659 9761 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f6115e1ca6b465eabb09830e8389037 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f6115e1ca6b465eabb09830e8389037" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 38763 } }
I20260812 16:41:42.677726 9761 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f6115e1ca6b465eabb09830e8389037 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:41:42.677747 9761 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f6115e1ca6b465eabb09830e8389037 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:41:42.677819 9761 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f6115e1ca6b465eabb09830e8389037 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:42.678086 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 9658
I20260812 16:41:42.678231 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:41:42.678411 1015 external_mini_cluster.cc:1428] Setting key 3d364ad08f6637e78d47292b8f877bff
I20260812 16:41:42.678855 9761 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f6115e1ca6b465eabb09830e8389037 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f6115e1ca6b465eabb09830e8389037" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 38763 } }
I20260812 16:41:42.678972 9761 leader_election.cc:302] T 00000000000000000000000000000000 P 1f6115e1ca6b465eabb09830e8389037 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f6115e1ca6b465eabb09830e8389037; no voters:
I20260812 16:41:42.679183 9761 leader_election.cc:288] T 00000000000000000000000000000000 P 1f6115e1ca6b465eabb09830e8389037 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:41:42.679206 9774 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f6115e1ca6b465eabb09830e8389037 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:41:42.679368 9774 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f6115e1ca6b465eabb09830e8389037 [term 1 LEADER]: Becoming Leader. State: Replica: 1f6115e1ca6b465eabb09830e8389037, State: Running, Role: LEADER
I20260812 16:41:42.679531 9761 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f6115e1ca6b465eabb09830e8389037 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:41:42.679507 9774 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f6115e1ca6b465eabb09830e8389037 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f6115e1ca6b465eabb09830e8389037" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 38763 } }
I20260812 16:41:42.679977 9774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f6115e1ca6b465eabb09830e8389037 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f6115e1ca6b465eabb09830e8389037. Latest consensus state: current_term: 1 leader_uuid: "1f6115e1ca6b465eabb09830e8389037" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f6115e1ca6b465eabb09830e8389037" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 38763 } } }
I20260812 16:41:42.680125 9774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f6115e1ca6b465eabb09830e8389037 [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:42.680307 9776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f6115e1ca6b465eabb09830e8389037 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f6115e1ca6b465eabb09830e8389037" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f6115e1ca6b465eabb09830e8389037" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 38763 } } }
I20260812 16:41:42.680415 9795 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:41:42.680452 9776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f6115e1ca6b465eabb09830e8389037 [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:42.681057 9795 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:41:42.684315 9795 catalog_manager.cc:1347] Generated new cluster ID: 907645357a26412d9b930596f69f3bd6
I20260812 16:41:42.684370 9795 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 16:41:42 dist-test-slave-w4v5 krb5kdc[9637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:41:42.690742 9795 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:41:42.691319 9773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:42.678921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41905 (local address 127.0.253.254:38763)
0812 16:41:42.679132 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:42.679142 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:42.679182 (+ 40us) server_negotiation.cc:407] Connection header received
0812 16:41:42.679309 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:42.679317 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:42.679379 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:42.679588 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:42.680281 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:42.681653 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:42.682597 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:42.683229 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:42.683351 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:42.686312 (+ 2961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:42.686317 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:42.686317 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:42.686331 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:42.689145 (+ 2814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:42.689575 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:42.689578 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:42.689578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:42.689711 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:42.689933 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:42.689934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:42.689935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:42.690250 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:42.690420 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:42.690908 (+ 488us) server_negotiation.cc:299] Negotiation successful
0812 16:41:42.691054 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":91,"threads_started":1}
I20260812 16:41:42.691468 9795 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:41:42.692276 1015 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:41:42.695055 9795 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f6115e1ca6b465eabb09830e8389037: Generated new TSK 0
I20260812 16:41:42.695238 9795 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:41:42.700341 9773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:42.693821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35715 (local address 127.0.253.254:38763)
0812 16:41:42.693857 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:42.693859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:42.693962 (+ 103us) server_negotiation.cc:407] Connection header received
0812 16:41:42.694007 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:42.694007 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:42.694017 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:42.694084 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:42.694683 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:42.695487 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:42.696212 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:42.696544 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:42.696615 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:42.697758 (+ 1143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:42.697760 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:42.697760 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:42.697766 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:42.699289 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:42.699630 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:42.699631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:42.699631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:42.699691 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:42.699871 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:42.699872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:42.699872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:42.699997 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:42.700099 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:42.700177 (+ 78us) server_negotiation.cc:299] Negotiation successful
0812 16:41:42.700255 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 16:41:42.702272 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 9658
2026-08-12T16:41:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1602 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:41:42 dist-test-slave-w4v5 krb5kdc[9820](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:41:42 dist-test-slave-w4v5 krb5kdc[9820](info): set up 2 sockets
Aug 12 16:41:42 dist-test-slave-w4v5 krb5kdc[9820](info): commencing operation
krb5kdc: starting...
W20260812 16:41:44.014178 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.286s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:41:44 dist-test-slave-w4v5 krb5kdc[9820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:41:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:41:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
I20260812 16:41:44.146152 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:37067
--webserver_interface=127.0.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:32857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:37067
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:41:44.230713 9841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:44.231159 9841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:44.231201 9841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:44.232620 9841 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:41:44.232733 9841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:44.232755 9841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:44.232769 9841 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:41:44.232781 9841 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:41:44.234340 9841 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:32857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:37067
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:37067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:44.234608 9841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:44.234879 9841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:44.239979 9848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:44.240141 9841 server_base.cc:1034] running on GCE node
W20260812 16:41:44.239965 9849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:44.239959 9853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:44.240458 9841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:44.240813 9841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:44.241986 9841 hybrid_clock.cc:648] HybridClock initialized: now 1786552904241959 us; error 48 us; skew 500 ppm
Aug 12 16:41:44 dist-test-slave-w4v5 krb5kdc[9820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552904, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:41:44.244702 9841 init.cc:377] Logged in from keytab as kudu/127.0.253.254@KRBTEST.COM (short username kudu)
I20260812 16:41:44.245167 9841 webserver.cc:460] Webserver started at http://127.0.253.254:38205/ using document root <none> and password file <none>
I20260812 16:41:44.245378 9841 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:44.245422 9841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:44.245513 9841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:44.246549 9841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "5ed500bb722f441b870cf761e6676237"
format_stamp: "Formatted at 2026-08-12 16:41:44 on dist-test-slave-w4v5"
server_key: "6809758376467d2f04f849d23543024d"
server_key_iv: "575edc02ece3f4bc55e185882738ad3c"
server_key_version: "encryptionkey@0"
I20260812 16:41:44.246884 9841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "5ed500bb722f441b870cf761e6676237"
format_stamp: "Formatted at 2026-08-12 16:41:44 on dist-test-slave-w4v5"
server_key: "6809758376467d2f04f849d23543024d"
server_key_iv: "575edc02ece3f4bc55e185882738ad3c"
server_key_version: "encryptionkey@0"
I20260812 16:41:44.248306 9841 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:41:44.249030 9867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:44.249236 9841 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:41:44.249334 9841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "5ed500bb722f441b870cf761e6676237"
format_stamp: "Formatted at 2026-08-12 16:41:44 on dist-test-slave-w4v5"
server_key: "6809758376467d2f04f849d23543024d"
server_key_iv: "575edc02ece3f4bc55e185882738ad3c"
server_key_version: "encryptionkey@0"
I20260812 16:41:44.249393 9841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:44.279563 9841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:44.281802 9841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:44.281930 9841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:44.286065 9841 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:37067
I20260812 16:41:44.286129 9966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:37067 every 8 connection(s)
I20260812 16:41:44.286489 9841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:41:44.287590 9967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:44.290021 9967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ed500bb722f441b870cf761e6676237: Bootstrap starting.
I20260812 16:41:44.290673 9967 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ed500bb722f441b870cf761e6676237: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:44.290993 9967 log.cc:824] T 00000000000000000000000000000000 P 5ed500bb722f441b870cf761e6676237: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:44.291795 9967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ed500bb722f441b870cf761e6676237: No bootstrap required, opened a new log
I20260812 16:41:44.291903 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 9841
I20260812 16:41:44.292030 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:41:44.292207 1015 external_mini_cluster.cc:1428] Setting key 42235fa95c6c57052ed263f81f692867
I20260812 16:41:44.293934 9967 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ed500bb722f441b870cf761e6676237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ed500bb722f441b870cf761e6676237" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 37067 } }
I20260812 16:41:44.294118 9967 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ed500bb722f441b870cf761e6676237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:44.294160 9967 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ed500bb722f441b870cf761e6676237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ed500bb722f441b870cf761e6676237, State: Initialized, Role: FOLLOWER
I20260812 16:41:44.294298 9967 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ed500bb722f441b870cf761e6676237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ed500bb722f441b870cf761e6676237" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 37067 } }
I20260812 16:41:44.294400 9967 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ed500bb722f441b870cf761e6676237 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:41:44.294471 9967 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ed500bb722f441b870cf761e6676237 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:41:44.294525 9967 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ed500bb722f441b870cf761e6676237 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:44.295199 9967 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ed500bb722f441b870cf761e6676237 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ed500bb722f441b870cf761e6676237" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 37067 } }
I20260812 16:41:44.295295 9967 leader_election.cc:302] T 00000000000000000000000000000000 P 5ed500bb722f441b870cf761e6676237 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ed500bb722f441b870cf761e6676237; no voters:
I20260812 16:41:44.295481 9967 leader_election.cc:288] T 00000000000000000000000000000000 P 5ed500bb722f441b870cf761e6676237 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:41:44.295508 9972 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ed500bb722f441b870cf761e6676237 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:41:44.295631 9972 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ed500bb722f441b870cf761e6676237 [term 1 LEADER]: Becoming Leader. State: Replica: 5ed500bb722f441b870cf761e6676237, State: Running, Role: LEADER
I20260812 16:41:44.295719 9972 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ed500bb722f441b870cf761e6676237 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ed500bb722f441b870cf761e6676237" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 37067 } }
I20260812 16:41:44.295776 9967 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ed500bb722f441b870cf761e6676237 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:41:44.296072 9972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ed500bb722f441b870cf761e6676237 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ed500bb722f441b870cf761e6676237. Latest consensus state: current_term: 1 leader_uuid: "5ed500bb722f441b870cf761e6676237" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ed500bb722f441b870cf761e6676237" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 37067 } } }
I20260812 16:41:44.296145 9972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ed500bb722f441b870cf761e6676237 [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:44.296103 9973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ed500bb722f441b870cf761e6676237 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5ed500bb722f441b870cf761e6676237" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ed500bb722f441b870cf761e6676237" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 37067 } } }
I20260812 16:41:44.296331 9973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ed500bb722f441b870cf761e6676237 [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:44.296644 9978 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:41:44.297222 9978 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:41:44.299396 9971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:44.292843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59477 (local address 127.0.253.254:37067)
0812 16:41:44.293044 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:44.293052 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:44.293078 (+ 26us) server_negotiation.cc:407] Connection header received
0812 16:41:44.293908 (+ 830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:44.293915 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:44.293965 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:44.294088 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:44.294406 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:44.296038 (+ 1632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:44.297217 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:44.297824 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:44.297906 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:44.298634 (+ 728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:44.298638 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:44.298639 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:41:44.298839 (+ 200us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:44.298872 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:41:44.298883 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:44.298937 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:44.299052 (+ 115us) server_negotiation.cc:299] Negotiation successful
0812 16:41:44.299167 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":110,"threads_started":1}
I20260812 16:41:44.300513 1015 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:41:44.305092 9971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:44.301894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37881 (local address 127.0.253.254:37067)
0812 16:41:44.301929 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:44.301931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:44.302087 (+ 156us) server_negotiation.cc:407] Connection header received
0812 16:41:44.302160 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:44.302161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:44.302172 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:44.302217 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:44.302329 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:44.302793 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:44.303804 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:44.304055 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:44.304090 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:44.304847 (+ 757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:44.304848 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:44.304849 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:41:44.304853 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:44.304868 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:41:44.304872 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:44.304928 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:44.304987 (+ 59us) server_negotiation.cc:299] Negotiation successful
0812 16:41:44.305010 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260812 16:41:44.307552 9978 catalog_manager.cc:1347] Generated new cluster ID: c19fa8811d3540f3a3db5cabed2e9517
I20260812 16:41:44.307617 9978 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:41:44.327963 9978 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:41:44.328523 9978 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:41:44.331768 9978 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5ed500bb722f441b870cf761e6676237: Generated new TSK 0
I20260812 16:41:44.332120 9978 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:41:44.337744 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 9841
2026-08-12T16:41:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1637 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:41:44 dist-test-slave-w4v5 krb5kdc[10019](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:41:44 dist-test-slave-w4v5 krb5kdc[10019](info): set up 2 sockets
Aug 12 16:41:44 dist-test-slave-w4v5 krb5kdc[10019](info): commencing operation
krb5kdc: starting...
W20260812 16:41:45.657337 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.292s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:41:45 dist-test-slave-w4v5 krb5kdc[10019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:41:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:41:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
I20260812 16:41:45.790143 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:44981
--webserver_interface=127.0.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:32835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:44981
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:41:45.877140 10044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:45.877362 10044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:45.877386 10044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:45.878553 10044 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:41:45.878599 10044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:45.878613 10044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:45.878626 10044 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:41:45.878638 10044 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:41:45.880321 10044 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:32835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:44981
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:44981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:45.880599 10044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:45.880841 10044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:45.886083 10053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:45.886091 10054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:45.886097 10056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:45.886374 10044 server_base.cc:1034] running on GCE node
I20260812 16:41:45.886730 10044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:45.887012 10044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:45.888180 10044 hybrid_clock.cc:648] HybridClock initialized: now 1786552905888153 us; error 44 us; skew 500 ppm
Aug 12 16:41:45 dist-test-slave-w4v5 krb5kdc[10019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552905, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:41:45.891006 10044 init.cc:377] Logged in from keytab as kudu/127.0.253.254@KRBTEST.COM (short username kudu)
I20260812 16:41:45.891543 10044 webserver.cc:460] Webserver started at http://127.0.253.254:34437/ using document root <none> and password file <none>
I20260812 16:41:45.891779 10044 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:45.891849 10044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:45.892007 10044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:45.893148 10044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "74c422ec6b514c628d12b1820817c600"
format_stamp: "Formatted at 2026-08-12 16:41:45 on dist-test-slave-w4v5"
server_key: "83d3f30332d4e1021d732fa2f5a2cefa"
server_key_iv: "0223975010d65cb90f228e4d349802b9"
server_key_version: "encryptionkey@0"
I20260812 16:41:45.893572 10044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "74c422ec6b514c628d12b1820817c600"
format_stamp: "Formatted at 2026-08-12 16:41:45 on dist-test-slave-w4v5"
server_key: "83d3f30332d4e1021d732fa2f5a2cefa"
server_key_iv: "0223975010d65cb90f228e4d349802b9"
server_key_version: "encryptionkey@0"
I20260812 16:41:45.894979 10044 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:41:45.895867 10067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:45.896054 10044 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:41:45.896173 10044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "74c422ec6b514c628d12b1820817c600"
format_stamp: "Formatted at 2026-08-12 16:41:45 on dist-test-slave-w4v5"
server_key: "83d3f30332d4e1021d732fa2f5a2cefa"
server_key_iv: "0223975010d65cb90f228e4d349802b9"
server_key_version: "encryptionkey@0"
I20260812 16:41:45.896252 10044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:45.899973 10044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:45.902069 10044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:45.902256 10044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:45.906599 10044 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:44981
I20260812 16:41:45.906625 10165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:44981 every 8 connection(s)
I20260812 16:41:45.906988 10044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:41:45.908094 10166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:45.910434 10166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74c422ec6b514c628d12b1820817c600: Bootstrap starting.
I20260812 16:41:45.911036 10166 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74c422ec6b514c628d12b1820817c600: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:45.911370 10166 log.cc:824] T 00000000000000000000000000000000 P 74c422ec6b514c628d12b1820817c600: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:45.912146 10166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74c422ec6b514c628d12b1820817c600: No bootstrap required, opened a new log
I20260812 16:41:45.913767 10166 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74c422ec6b514c628d12b1820817c600 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74c422ec6b514c628d12b1820817c600" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 44981 } }
I20260812 16:41:45.913920 10166 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74c422ec6b514c628d12b1820817c600 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:41:45.913944 10166 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74c422ec6b514c628d12b1820817c600 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74c422ec6b514c628d12b1820817c600, State: Initialized, Role: FOLLOWER
I20260812 16:41:45.914105 10166 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74c422ec6b514c628d12b1820817c600 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74c422ec6b514c628d12b1820817c600" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 44981 } }
I20260812 16:41:45.914201 10166 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74c422ec6b514c628d12b1820817c600 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:41:45.914224 10166 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74c422ec6b514c628d12b1820817c600 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:41:45.914263 10166 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74c422ec6b514c628d12b1820817c600 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:45.915175 10166 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74c422ec6b514c628d12b1820817c600 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74c422ec6b514c628d12b1820817c600" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 44981 } }
I20260812 16:41:45.915282 10166 leader_election.cc:302] T 00000000000000000000000000000000 P 74c422ec6b514c628d12b1820817c600 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74c422ec6b514c628d12b1820817c600; no voters:
I20260812 16:41:45.915409 10166 leader_election.cc:288] T 00000000000000000000000000000000 P 74c422ec6b514c628d12b1820817c600 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:41:45.915467 10174 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74c422ec6b514c628d12b1820817c600 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:41:45.915681 10166 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74c422ec6b514c628d12b1820817c600 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:41:45.915721 10174 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74c422ec6b514c628d12b1820817c600 [term 1 LEADER]: Becoming Leader. State: Replica: 74c422ec6b514c628d12b1820817c600, State: Running, Role: LEADER
I20260812 16:41:45.915890 10174 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74c422ec6b514c628d12b1820817c600 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74c422ec6b514c628d12b1820817c600" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 44981 } }
I20260812 16:41:45.916204 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 10044
I20260812 16:41:45.916326 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:41:45.916288 10175 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74c422ec6b514c628d12b1820817c600 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74c422ec6b514c628d12b1820817c600" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74c422ec6b514c628d12b1820817c600" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 44981 } } }
I20260812 16:41:45.916363 10175 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74c422ec6b514c628d12b1820817c600 [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:45.916534 1015 external_mini_cluster.cc:1428] Setting key a9f9d92918fecb2837590588df88e4d0
I20260812 16:41:45.916874 10176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74c422ec6b514c628d12b1820817c600 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74c422ec6b514c628d12b1820817c600. Latest consensus state: current_term: 1 leader_uuid: "74c422ec6b514c628d12b1820817c600" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74c422ec6b514c628d12b1820817c600" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 44981 } } }
I20260812 16:41:45.917088 10189 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:41:45.917367 10176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74c422ec6b514c628d12b1820817c600 [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:45.918115 10189 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:41:45.919528 10201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:45.917234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49037 (local address 127.0.253.254:44981)
0812 16:41:45.917716 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:45.917728 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:45.917763 (+ 35us) server_negotiation.cc:407] Connection header received
0812 16:41:45.917891 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:45.917899 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:45.917932 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:45.918080 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:45.919000 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:45.919004 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:45.919004 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:41:45.919009 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:45.919060 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:41:45.919093 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:45.919165 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:45.919300 (+ 135us) server_negotiation.cc:299] Negotiation successful
0812 16:41:45.919346 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":73,"threads_started":1}
I20260812 16:41:45.919939 10189 catalog_manager.cc:1347] Generated new cluster ID: 285b68c779244c1ab2a0bc305837f98b
I20260812 16:41:45.919998 10189 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:41:45.920569 1015 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:41:45.923470 10201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:45.922182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41163 (local address 127.0.253.254:44981)
0812 16:41:45.922219 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:45.922221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:45.922369 (+ 148us) server_negotiation.cc:407] Connection header received
0812 16:41:45.922474 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:45.922475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:45.922480 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:45.922512 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:45.923218 (+ 706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:45.923220 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:45.923220 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:41:45.923225 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:45.923249 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:41:45.923254 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:45.923286 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:45.923396 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 16:41:45.923409 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 16:41:45.942112 10189 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:41:45.942313 10189 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:41:45.945187 10189 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74c422ec6b514c628d12b1820817c600: Generated new TSK 0
I20260812 16:41:45.945369 10189 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:41:45.949291 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 10044
2026-08-12T16:41:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1622 ms)
[----------] 18 tests from AuthTokenIssuingTest (29382 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:41:46 dist-test-slave-w4v5 krb5kdc[10224](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:41:46 dist-test-slave-w4v5 krb5kdc[10224](info): set up 2 sockets
Aug 12 16:41:46 dist-test-slave-w4v5 krb5kdc[10224](info): commencing operation
krb5kdc: starting...
W20260812 16:41:47.312036 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.323s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:41:47 dist-test-slave-w4v5 krb5kdc[10224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:41:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:41:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
I20260812 16:41:47.440291 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:36593
--webserver_interface=127.0.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:33051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:36593,127.0.253.253:36563,127.0.253.252:39587
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:41:47.524435 10246 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 16:41:47.524705 10246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:47.524727 10246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:47.524750 10246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:47.525985 10246 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:41:47.526036 10246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:47.526052 10246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:47.526067 10246 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:41:47.526081 10246 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:41:47.527817 10246 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:33051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:36593,127.0.253.253:36563,127.0.253.252:39587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:36593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:47.528039 10246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:47.528231 10246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:47.533723 10254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:47.533831 10264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:47.533730 10257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:47.534302 10246 server_base.cc:1034] running on GCE node
I20260812 16:41:47.534605 10246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:47.534906 10246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:47.536059 10246 hybrid_clock.cc:648] HybridClock initialized: now 1786552907536047 us; error 31 us; skew 500 ppm
Aug 12 16:41:47 dist-test-slave-w4v5 krb5kdc[10224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552907, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:41:47.541935 10246 init.cc:377] Logged in from keytab as kudu/127.0.253.254@KRBTEST.COM (short username kudu)
I20260812 16:41:47.542507 10246 webserver.cc:460] Webserver started at http://127.0.253.254:44455/ using document root <none> and password file <none>
I20260812 16:41:47.542778 10246 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:47.542858 10246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:47.543030 10246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:47.544273 10246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "df35c5527cf545049882c590161af7a1"
format_stamp: "Formatted at 2026-08-12 16:41:47 on dist-test-slave-w4v5"
server_key: "7e1326b3feccab39148d4d1447ae4144"
server_key_iv: "deae0b743655d5380062f0f49f7530db"
server_key_version: "encryptionkey@0"
I20260812 16:41:47.544688 10246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "df35c5527cf545049882c590161af7a1"
format_stamp: "Formatted at 2026-08-12 16:41:47 on dist-test-slave-w4v5"
server_key: "7e1326b3feccab39148d4d1447ae4144"
server_key_iv: "deae0b743655d5380062f0f49f7530db"
server_key_version: "encryptionkey@0"
I20260812 16:41:47.546209 10246 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:41:47.547047 10277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:47.547303 10246 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:41:47.547422 10246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "df35c5527cf545049882c590161af7a1"
format_stamp: "Formatted at 2026-08-12 16:41:47 on dist-test-slave-w4v5"
server_key: "7e1326b3feccab39148d4d1447ae4144"
server_key_iv: "deae0b743655d5380062f0f49f7530db"
server_key_version: "encryptionkey@0"
I20260812 16:41:47.547516 10246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:47.566999 10246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:47.569789 10246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:47.569980 10246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:47.574821 10246 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:36593
I20260812 16:41:47.574864 10359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:36593 every 8 connection(s)
I20260812 16:41:47.575296 10246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:41:47.576169 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 10246
I20260812 16:41:47.576305 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:41:47.576503 1015 external_mini_cluster.cc:1428] Setting key 54390c99d4e681133ea7673e6d846b6e
I20260812 16:41:47.576507 10361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:47.578146 10361 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36593 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 16:41:47 dist-test-slave-w4v5 krb5kdc[10224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552907, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.253.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.253@KRBTEST.COM" created.
I20260812 16:41:47.588704 10365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:47.578968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59244 (local address 127.0.253.254:36593)
0812 16:41:47.579243 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:47.579252 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:47.579396 (+ 144us) server_negotiation.cc:407] Connection header received
0812 16:41:47.580090 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:47.580092 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:47.580113 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:47.580229 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:47.580793 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.581622 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.582548 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.582881 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.583017 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:47.585241 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:47.585251 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:47.585251 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:47.585259 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:47.586824 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:47.587300 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:47.587303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:47.587303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:47.587432 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:47.587709 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:47.587710 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:47.587711 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:47.587990 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:47.588096 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:47.588308 (+ 212us) server_negotiation.cc:299] Negotiation successful
0812 16:41:47.588451 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":175,"threads_started":1}
I20260812 16:41:47.588704 10364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:47.578932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36593 (local address 127.0.0.1:59244)
0812 16:41:47.579190 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:47.579242 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:47.579888 (+ 646us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:47.580237 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:47.580239 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:47.580526 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:41:47.580684 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:47.580689 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.581750 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.581751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:47.582438 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:47.582440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.583012 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.583013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:47.583112 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:47.583694 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:41:47.583703 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:41:47.585145 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:41:47.586916 (+ 1771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:47.586924 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:47.586926 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:47.587211 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:47.587514 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:47.587515 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:47.587515 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:47.587628 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:47.588072 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:41:47.588079 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:41:47.588144 (+ 65us) client_negotiation.cc:769] Sending connection context
0812 16:41:47.588260 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 16:41:47.588518 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":107,"thread_start_us":168,"threads_started":1}
I20260812 16:41:47.590312 10361 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.253.253" port: 36563 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:41:47.590703 10364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:47.590579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.253:36563 (local address 127.0.0.1:43026)
0812 16:41:47.590634 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:47.590680 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.253:36563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 16:41:47.590843 10280 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.253.253:36563: connect: Connection refused (error 111)
I20260812 16:41:47.591302 10364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:47.591199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.253:36563 (local address 127.0.0.1:43030)
0812 16:41:47.591253 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:47.591279 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.253:36563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 16:41:47.591485 10361 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.253.253:36563: Network error: Client connection negotiation failed: client connection to 127.0.253.253:36563: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.253.keytab.
Entry for principal kudu/127.0.253.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.253.keytab.
I20260812 16:41:47.602975 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.253:36563
--webserver_interface=127.0.253.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.253.keytab
--principal=kudu/127.0.253.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:33051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:36593,127.0.253.253:36563,127.0.253.252:39587
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:41:47.640787 10361 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.253.253" port: 36563 } attempt: 1
I20260812 16:41:47.641427 10364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:47.641268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.253:36563 (local address 127.0.0.1:43032)
0812 16:41:47.641351 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:47.641394 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.253:36563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260812 16:41:47.641824 10364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:47.641714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.253:36563 (local address 127.0.0.1:43042)
0812 16:41:47.641767 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:47.641803 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.253:36563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 16:41:47.641924 10361 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.253.253:36563: Network error: Client connection negotiation failed: client connection to 127.0.253.253:36563: connect: Connection refused (error 111)
W20260812 16:41:47.688115 10369 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 16:41:47.688328 10369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:47.688345 10369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:47.688390 10369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:47.689513 10369 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:41:47.689555 10369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:47.689570 10369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:47.689615 10369 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:41:47.689644 10369 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:41:47.691197 10369 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:33051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:36593,127.0.253.253:36563,127.0.253.252:39587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.253.keytab
--principal=kudu/127.0.253.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.253:36563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.253.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:47.691459 10369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:47.691702 10369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:47.696722 10381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:47.696722 10386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:47.696770 10383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:47.696897 10369 server_base.cc:1034] running on GCE node
I20260812 16:41:47.697276 10369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:47.697523 10369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:47.698673 10369 hybrid_clock.cc:648] HybridClock initialized: now 1786552907698677 us; error 45 us; skew 500 ppm
Aug 12 16:41:47 dist-test-slave-w4v5 krb5kdc[10224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552907, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:41:47.701382 10369 init.cc:377] Logged in from keytab as kudu/127.0.253.253@KRBTEST.COM (short username kudu)
I20260812 16:41:47.701871 10369 webserver.cc:460] Webserver started at http://127.0.253.253:37351/ using document root <none> and password file <none>
I20260812 16:41:47.702064 10369 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:47.702114 10369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:47.702214 10369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:47.703305 10369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-1/data/instance:
uuid: "2cf47406684346a39e4733f2aa758126"
format_stamp: "Formatted at 2026-08-12 16:41:47 on dist-test-slave-w4v5"
server_key: "97acf5e59ad58855bb2cae6f7b395c15"
server_key_iv: "c79e93387bbe8f15955a95867b067159"
server_key_version: "encryptionkey@0"
I20260812 16:41:47.703625 10369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-1/wal/instance:
uuid: "2cf47406684346a39e4733f2aa758126"
format_stamp: "Formatted at 2026-08-12 16:41:47 on dist-test-slave-w4v5"
server_key: "97acf5e59ad58855bb2cae6f7b395c15"
server_key_iv: "c79e93387bbe8f15955a95867b067159"
server_key_version: "encryptionkey@0"
I20260812 16:41:47.704957 10369 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:41:47.705689 10403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:47.705885 10369 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:41:47.705986 10369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-1/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-1/wal
uuid: "2cf47406684346a39e4733f2aa758126"
format_stamp: "Formatted at 2026-08-12 16:41:47 on dist-test-slave-w4v5"
server_key: "97acf5e59ad58855bb2cae6f7b395c15"
server_key_iv: "c79e93387bbe8f15955a95867b067159"
server_key_version: "encryptionkey@0"
I20260812 16:41:47.706061 10369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:47.710076 10361 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.253.253" port: 36563 } attempt: 2
I20260812 16:41:47.710512 10364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:47.710377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.253:36563 (local address 127.0.0.1:43056)
0812 16:41:47.710437 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:47.710474 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.253:36563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 16:41:47.710963 10364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:47.710865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.253:36563 (local address 127.0.0.1:43072)
0812 16:41:47.710909 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:47.710935 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.253:36563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260812 16:41:47.711102 10361 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.253.253:36563: Network error: Client connection negotiation failed: client connection to 127.0.253.253:36563: connect: Connection refused (error 111)
I20260812 16:41:47.717028 10369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:47.719502 10369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:47.719619 10369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:47.723946 10369 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.253:36563
I20260812 16:41:47.724002 10490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.253:36563 every 8 connection(s)
I20260812 16:41:47.724371 10369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-1/data/info.pb
I20260812 16:41:47.725394 10493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:47.726967 10493 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36593 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:41:47.728706 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 10369
I20260812 16:41:47.728823 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-1/wal/instance
I20260812 16:41:47.729014 1015 external_mini_cluster.cc:1428] Setting key bd86dfcfb0ffa27f910684455113763f
Aug 12 16:41:47 dist-test-slave-w4v5 krb5kdc[10224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552907, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.253@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:41:47.737824 10365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:47.727735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59252 (local address 127.0.253.254:36593)
0812 16:41:47.727821 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:47.727824 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:47.728002 (+ 178us) server_negotiation.cc:407] Connection header received
0812 16:41:47.728462 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:47.728463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:47.728476 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:47.728521 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:47.729442 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.730275 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.731721 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.732048 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.732139 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:47.734789 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:47.734791 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:47.734791 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:47.734798 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:47.736274 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:47.736818 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:47.736819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:47.736819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:47.736874 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:47.737176 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:47.737176 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:47.737177 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:47.737283 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:47.737334 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:47.737686 (+ 352us) server_negotiation.cc:299] Negotiation successful
0812 16:41:47.737759 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 16:41:47.738103 10500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:47.727691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36593 (local address 127.0.0.1:59252)
0812 16:41:47.727879 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:47.727919 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:47.728379 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:47.728605 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:47.728610 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:47.728987 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:41:47.729308 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:47.729314 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.730364 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.730366 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:47.731634 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:47.731637 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.732196 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.732197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:47.732321 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:47.732896 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:41:47.732905 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:41:47.734692 (+ 1787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:41:47.736361 (+ 1669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:47.736370 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:47.736395 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:47.736729 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:47.736946 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:47.736947 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:47.736948 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:47.737107 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:47.737354 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:41:47.737362 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:41:47.737541 (+ 179us) client_negotiation.cc:769] Sending connection context
0812 16:41:47.737668 (+ 127us) client_negotiation.cc:240] Negotiation successful
0812 16:41:47.737847 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":97,"threads_started":1}
I20260812 16:41:47.738759 10493 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.253.253" port: 36563 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.0.253.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.252@KRBTEST.COM" created.
Aug 12 16:41:47 dist-test-slave-w4v5 krb5kdc[10224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552907, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.253@KRBTEST.COM for kudu/127.0.253.253@KRBTEST.COM
I20260812 16:41:47.747391 10500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:47.739038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.253:36563 (local address 127.0.0.1:43088)
0812 16:41:47.739119 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:47.739128 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:47.739170 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:47.739499 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:47.739500 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:47.739661 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:41:47.739713 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:47.739714 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.740708 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.740709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:47.741322 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:47.741324 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.741983 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.741984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:47.742080 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:47.742452 (+ 372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:41:47.742458 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:41:47.744047 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:41:47.746330 (+ 2283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:47.746331 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:47.746332 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:47.746492 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:47.746738 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:47.746739 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:47.746739 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:47.746790 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:47.747162 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:41:47.747162 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:41:47.747199 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 16:41:47.747253 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 16:41:47.747326 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260812 16:41:47.747556 10504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:47.739135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43088 (local address 127.0.253.253:36563)
0812 16:41:47.739349 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:47.739353 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:47.739377 (+ 24us) server_negotiation.cc:407] Connection header received
0812 16:41:47.739397 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:47.739399 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:47.739419 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:47.739499 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:47.739793 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.740610 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.741395 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.741878 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.741956 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:47.744142 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:47.744150 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:47.744150 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:47.744156 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:47.746235 (+ 2079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:47.746582 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:47.746587 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:47.746588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:47.746662 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:47.746854 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:47.746855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:47.746856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:47.747097 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:47.747192 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:47.747300 (+ 108us) server_negotiation.cc:299] Negotiation successful
0812 16:41:47.747393 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":111,"threads_started":1}
I20260812 16:41:47.748211 10493 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.253.252" port: 39587 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:41:47.748584 10500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:47.748467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:39587 (local address 127.0.0.1:53934)
0812 16:41:47.748514 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:47.748564 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.252:39587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260812 16:41:47.748770 10405 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.253.252:39587: connect: Connection refused (error 111)
I20260812 16:41:47.749261 10500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:47.749136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:39587 (local address 127.0.0.1:53940)
0812 16:41:47.749196 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:47.749232 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.252:39587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 16:41:47.749423 10493 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.253.252:39587: Network error: Client connection negotiation failed: client connection to 127.0.253.252:39587: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.252.keytab.
Entry for principal kudu/127.0.253.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.252.keytab.
I20260812 16:41:47.758185 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.252:39587
--webserver_interface=127.0.253.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.252.keytab
--principal=kudu/127.0.253.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:33051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:36593,127.0.253.253:36563,127.0.253.252:39587
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:41:47.798684 10493 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.253.252" port: 39587 } attempt: 1
I20260812 16:41:47.799347 10500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:47.799192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:39587 (local address 127.0.0.1:53952)
0812 16:41:47.799260 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:47.799312 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.252:39587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 16:41:47.799796 10500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:47.799689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:39587 (local address 127.0.0.1:53954)
0812 16:41:47.799742 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:47.799775 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.252:39587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 16:41:47.799937 10493 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.253.252:39587: Network error: Client connection negotiation failed: client connection to 127.0.253.252:39587: connect: Connection refused (error 111)
I20260812 16:41:47.802253 10361 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.253.253" port: 36563 } attempt: 3
Aug 12 16:41:47 dist-test-slave-w4v5 krb5kdc[10224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552907, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for kudu/127.0.253.253@KRBTEST.COM
I20260812 16:41:47.809926 10504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:47.802707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43096 (local address 127.0.253.253:36563)
0812 16:41:47.802749 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:47.802751 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:47.802770 (+ 19us) server_negotiation.cc:407] Connection header received
0812 16:41:47.802796 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:47.802797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:47.802807 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:47.802864 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:47.803335 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.803858 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.804858 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.805081 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.805134 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:47.807203 (+ 2069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:47.807205 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:47.807205 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:47.807212 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:47.808746 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:47.809109 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:47.809111 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:47.809111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:47.809174 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:47.809382 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:47.809383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:47.809384 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:47.809498 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:47.809576 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:47.809751 (+ 175us) server_negotiation.cc:299] Negotiation successful
0812 16:41:47.809829 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 16:41:47.809927 10364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:47.802623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.253:36563 (local address 127.0.0.1:43096)
0812 16:41:47.802682 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:47.802693 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:47.802737 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:47.802894 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:47.802895 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:47.803154 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:41:47.803244 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:47.803246 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.803944 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.803946 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:47.804775 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:47.804778 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.805165 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.805166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:47.805255 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:47.805690 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:41:47.805696 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:41:47.807066 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:41:47.808840 (+ 1774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:47.808841 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:47.808841 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:47.809036 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:47.809249 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:47.809250 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:47.809250 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:47.809317 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:47.809569 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:41:47.809570 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:41:47.809618 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 16:41:47.809725 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 16:41:47.809829 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 16:41:47.810500 10361 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.253.252" port: 39587 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:41:47.810895 10364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:47.810778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:39587 (local address 127.0.0.1:53970)
0812 16:41:47.810833 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:47.810868 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.252:39587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 16:41:47.811386 10364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:47.811289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:39587 (local address 127.0.0.1:53978)
0812 16:41:47.811337 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:47.811366 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.252:39587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 16:41:47.811486 10361 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.253.252:39587: Network error: Client connection negotiation failed: client connection to 127.0.253.252:39587: connect: Connection refused (error 111)
I20260812 16:41:47.842666 10361 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.253.252" port: 39587 } attempt: 1
I20260812 16:41:47.843202 10364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:47.843032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:39587 (local address 127.0.0.1:53988)
0812 16:41:47.843116 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:47.843165 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.252:39587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260812 16:41:47.843617 10364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:47.843504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:39587 (local address 127.0.0.1:53996)
0812 16:41:47.843557 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:47.843593 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.252:39587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 16:41:47.843727 10361 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.253.252:39587: Network error: Client connection negotiation failed: client connection to 127.0.253.252:39587: connect: Connection refused (error 111)
W20260812 16:41:47.846606 10514 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 16:41:47.846801 10514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:47.846818 10514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:47.846885 10514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:47.848105 10514 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:41:47.848151 10514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:47.848165 10514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:47.848178 10514 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:41:47.848191 10514 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:41:47.849752 10514 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:33051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:36593,127.0.253.253:36563,127.0.253.252:39587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.252.keytab
--principal=kudu/127.0.253.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.252:39587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.253.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10514
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:47.850024 10514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:47.850286 10514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:47.855371 10523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:47.855378 10524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:47.855378 10529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:47.855502 10514 server_base.cc:1034] running on GCE node
I20260812 16:41:47.855924 10514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:47.856223 10514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:47.857455 10514 hybrid_clock.cc:648] HybridClock initialized: now 1786552907857426 us; error 39 us; skew 500 ppm
Aug 12 16:41:47 dist-test-slave-w4v5 krb5kdc[10224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552907, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:41:47.860214 10514 init.cc:377] Logged in from keytab as kudu/127.0.253.252@KRBTEST.COM (short username kudu)
I20260812 16:41:47.860721 10514 webserver.cc:460] Webserver started at http://127.0.253.252:35163/ using document root <none> and password file <none>
I20260812 16:41:47.860980 10514 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:47.861061 10514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:47.861239 10514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:47.862701 10514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-2/data/instance:
uuid: "07a261b7f5014d29b67dd6c3f0b7176d"
format_stamp: "Formatted at 2026-08-12 16:41:47 on dist-test-slave-w4v5"
server_key: "23f9d9d66fd9f7d712af0c81bdb33f07"
server_key_iv: "35712e58d1fe43458df8e94503831b1c"
server_key_version: "encryptionkey@0"
I20260812 16:41:47.863162 10514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-2/wal/instance:
uuid: "07a261b7f5014d29b67dd6c3f0b7176d"
format_stamp: "Formatted at 2026-08-12 16:41:47 on dist-test-slave-w4v5"
server_key: "23f9d9d66fd9f7d712af0c81bdb33f07"
server_key_iv: "35712e58d1fe43458df8e94503831b1c"
server_key_version: "encryptionkey@0"
I20260812 16:41:47.864563 10514 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:41:47.866057 10540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:47.866915 10514 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260812 16:41:47.867025 10514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-2/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-2/wal
uuid: "07a261b7f5014d29b67dd6c3f0b7176d"
format_stamp: "Formatted at 2026-08-12 16:41:47 on dist-test-slave-w4v5"
server_key: "23f9d9d66fd9f7d712af0c81bdb33f07"
server_key_iv: "35712e58d1fe43458df8e94503831b1c"
server_key_version: "encryptionkey@0"
I20260812 16:41:47.867126 10514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:47.868142 10493 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.253.252" port: 39587 } attempt: 2
I20260812 16:41:47.868577 10500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:47.868452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:39587 (local address 127.0.0.1:54012)
0812 16:41:47.868511 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:47.868544 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.252:39587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260812 16:41:47.868974 10500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:47.868850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:39587 (local address 127.0.0.1:54016)
0812 16:41:47.868901 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:47.868930 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.252:39587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 16:41:47.869064 10493 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.253.252:39587: Network error: Client connection negotiation failed: client connection to 127.0.253.252:39587: connect: Connection refused (error 111)
I20260812 16:41:47.884995 10514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:47.887212 10514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:47.887377 10514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:47.891805 10514 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.252:39587
I20260812 16:41:47.891844 10626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.252:39587 every 8 connection(s)
I20260812 16:41:47.892248 10514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-2/data/info.pb
I20260812 16:41:47.893405 10627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:47.894006 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 10514
I20260812 16:41:47.894120 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552619305671-1015-0/minicluster-data/master-2/wal/instance
I20260812 16:41:47.894289 1015 external_mini_cluster.cc:1428] Setting key 09d3f3fc45f3ddfd388526ab9799152d
I20260812 16:41:47.895516 10627 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36593 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 16:41:47 dist-test-slave-w4v5 krb5kdc[10224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.253.252@KRBTEST.COM
Aug 12 16:41:47 dist-test-slave-w4v5 krb5kdc[10224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552907, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.252@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:41:47.905052 10365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:47.896290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59268 (local address 127.0.253.254:36593)
0812 16:41:47.896358 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:47.896360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:47.896460 (+ 100us) server_negotiation.cc:407] Connection header received
0812 16:41:47.896522 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:47.896523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:47.896532 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:47.896583 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:47.898186 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.898630 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.899750 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.900086 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.900173 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:47.902219 (+ 2046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:47.902220 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:47.902221 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:47.902229 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:47.903649 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:47.904083 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:47.904085 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:47.904085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:47.904148 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:47.904410 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:47.904411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:47.904411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:47.904530 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:47.904616 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:47.904820 (+ 204us) server_negotiation.cc:299] Negotiation successful
0812 16:41:47.904912 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 16:41:47.905194 10631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:47.894866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54020 (local address 127.0.253.252:39587)
0812 16:41:47.895100 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:47.895113 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:47.895150 (+ 37us) server_negotiation.cc:407] Connection header received
0812 16:41:47.895601 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:47.895608 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:47.895658 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:47.895808 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:47.896605 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.897511 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.898491 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.899065 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.899199 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:47.901798 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:47.901807 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:47.901807 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:47.901815 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:47.903622 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:47.904019 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:47.904022 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:47.904023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:47.904110 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:47.904321 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:47.904322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:47.904323 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:47.904548 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:47.904703 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:47.904798 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 16:41:47.904971 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":144,"thread_start_us":87,"threads_started":1}
I20260812 16:41:47.905421 10639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:47.896177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:36593 (local address 127.0.0.1:59268)
0812 16:41:47.896332 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:47.896378 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:47.896455 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:47.897658 (+ 1203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:47.897661 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:47.898024 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:41:47.898107 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:47.898109 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.898716 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.898717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:47.899619 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:47.899622 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.900199 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.900200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:47.900255 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:47.900754 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:41:47.900763 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:41:47.902122 (+ 1359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:41:47.903756 (+ 1634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:47.903763 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:47.903765 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:47.903979 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:47.904235 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:47.904236 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:47.904237 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:47.904331 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:47.904607 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:41:47.904614 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:41:47.904692 (+ 78us) client_negotiation.cc:769] Sending connection context
0812 16:41:47.904797 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 16:41:47.905349 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":405,"thread_start_us":89,"threads_started":1}
I20260812 16:41:47.905943 10627 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.253.253" port: 36563 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 16:41:47 dist-test-slave-w4v5 krb5kdc[10224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552907, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.252@KRBTEST.COM for kudu/127.0.253.253@KRBTEST.COM
Aug 12 16:41:47 dist-test-slave-w4v5 krb5kdc[10224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:41:47.913177 10639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:47.906227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.253:36563 (local address 127.0.0.1:43108)
0812 16:41:47.906273 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:47.906284 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:47.906316 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:47.906489 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:47.906490 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:47.906632 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:41:47.906689 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:47.906691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.907410 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.907411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:47.907874 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:47.907876 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.908287 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.908288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:47.908341 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:47.908917 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:41:47.908922 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:41:47.910185 (+ 1263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:41:47.912138 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:47.912139 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:47.912140 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:47.912315 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:47.912524 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:47.912525 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:47.912526 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:47.912592 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:47.912852 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:41:47.912853 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:41:47.912896 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 16:41:47.912980 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 16:41:47.913087 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 16:41:47.913177 10504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:47.906318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43108 (local address 127.0.253.253:36563)
0812 16:41:47.906369 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:47.906370 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:47.906378 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:41:47.906401 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:47.906401 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:47.906410 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:47.906487 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:47.906780 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.907332 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.907990 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.908208 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.908279 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:47.910279 (+ 2000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:47.910281 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:47.910281 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:47.910289 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:47.912025 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:47.912391 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:47.912393 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:47.912393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:47.912456 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:47.912658 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:47.912659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:47.912659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:47.912776 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:47.912849 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:47.913007 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 16:41:47.913080 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 16:41:47.913504 10627 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.253.252" port: 39587 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:41:47.913902 10365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:47.906059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59282 (local address 127.0.253.254:36593)
0812 16:41:47.906100 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:47.906103 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:47.906112 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:41:47.906155 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:47.906156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:47.906166 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:47.906223 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:47.906681 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.907247 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.907941 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.908159 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.908211 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:47.910968 (+ 2757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:47.910969 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:47.910970 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:47.910976 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:47.912500 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:47.912863 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:47.912864 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:47.912864 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:47.912925 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:47.913126 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:47.913127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:47.913128 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:47.913283 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:47.913481 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:47.913695 (+ 214us) server_negotiation.cc:299] Negotiation successful
0812 16:41:47.913780 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
Aug 12 16:41:47 dist-test-slave-w4v5 krb5kdc[10224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552907, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.252@KRBTEST.COM for kudu/127.0.253.252@KRBTEST.COM
Aug 12 16:41:47 dist-test-slave-w4v5 krb5kdc[10224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.253.253@KRBTEST.COM
I20260812 16:41:47.918879 10361 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.253.252" port: 39587 } attempt: 2
I20260812 16:41:47.920529 10639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:47.913828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:39587 (local address 127.0.0.1:54022)
0812 16:41:47.913992 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:47.914003 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:47.914069 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:47.914199 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:47.914200 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:47.914345 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:41:47.914380 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:47.914382 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.915148 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.915148 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:47.915621 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:47.915622 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.916009 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.916009 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:47.916056 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:47.916446 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:41:47.916451 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:41:47.917658 (+ 1207us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:41:47.919417 (+ 1759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:47.919418 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:47.919418 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:47.919636 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:47.919837 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:47.919838 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:47.919839 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:47.919912 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:47.920227 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:41:47.920228 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:41:47.920264 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 16:41:47.920327 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 16:41:47.920417 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146}
I20260812 16:41:47.920898 10631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:47.913877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54022 (local address 127.0.253.252:39587)
0812 16:41:47.913913 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:47.913915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:47.914047 (+ 132us) server_negotiation.cc:407] Connection header received
0812 16:41:47.914139 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:47.914139 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:47.914148 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:47.914199 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:47.914444 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.915035 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.915716 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.915925 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.916008 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:47.917742 (+ 1734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:47.917743 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:47.917744 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:47.917750 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:47.919316 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:47.919697 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:47.919698 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:47.919699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:47.919756 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:47.919995 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:47.919996 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:47.919996 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:47.920135 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:47.920206 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:47.920719 (+ 513us) server_negotiation.cc:299] Negotiation successful
0812 16:41:47.920793 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 16:41:47.922597 10504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:47.914568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43124 (local address 127.0.253.253:36563)
0812 16:41:47.914606 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:47.914608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:47.914615 (+ 7us) server_negotiation.cc:407] Connection header received
0812 16:41:47.914640 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:47.914641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:47.914649 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:47.914709 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:47.915165 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.915609 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.916284 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.916606 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.916683 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:47.919552 (+ 2869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:47.919553 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:47.919554 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:47.919561 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:47.921364 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:47.921748 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:47.921749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:47.921750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:47.921811 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:47.922029 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:47.922030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:47.922030 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:47.922186 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:47.922349 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:47.922413 (+ 64us) server_negotiation.cc:299] Negotiation successful
0812 16:41:47.922495 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 16:41:47.922803 10627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07a261b7f5014d29b67dd6c3f0b7176d: Bootstrap starting.
I20260812 16:41:47.923640 10627 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 07a261b7f5014d29b67dd6c3f0b7176d: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:47.923990 10627 log.cc:824] T 00000000000000000000000000000000 P 07a261b7f5014d29b67dd6c3f0b7176d: Log is configured to *not* fsync() on all Append() calls
Aug 12 16:41:47 dist-test-slave-w4v5 krb5kdc[10224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552907, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for kudu/127.0.253.252@KRBTEST.COM
I20260812 16:41:47.924556 1015 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:41:47.924818 10627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07a261b7f5014d29b67dd6c3f0b7176d: No bootstrap required, opened a new log
I20260812 16:41:47.926056 10315 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "df35c5527cf545049882c590161af7a1"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59282
I20260812 16:41:47.927268 10627 raft_consensus.cc:348] T 00000000000000000000000000000000 P 07a261b7f5014d29b67dd6c3f0b7176d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df35c5527cf545049882c590161af7a1" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36593 } } peers { permanent_uuid: "2cf47406684346a39e4733f2aa758126" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 36563 } } peers { permanent_uuid: "07a261b7f5014d29b67dd6c3f0b7176d" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 39587 } }
I20260812 16:41:47.927443 10627 raft_consensus.cc:729] T 00000000000000000000000000000000 P 07a261b7f5014d29b67dd6c3f0b7176d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07a261b7f5014d29b67dd6c3f0b7176d, State: Initialized, Role: FOLLOWER
I20260812 16:41:47.927539 10627 consensus_queue.cc:260] T 00000000000000000000000000000000 P 07a261b7f5014d29b67dd6c3f0b7176d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df35c5527cf545049882c590161af7a1" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36593 } } peers { permanent_uuid: "2cf47406684346a39e4733f2aa758126" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 36563 } } peers { permanent_uuid: "07a261b7f5014d29b67dd6c3f0b7176d" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 39587 } }
I20260812 16:41:47.927865 10364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:47.919192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:39587 (local address 127.0.0.1:54036)
0812 16:41:47.919243 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:47.919256 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:47.919310 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:47.919609 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:47.919610 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:47.919781 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:41:47.919831 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:47.919833 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.921321 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.921322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:47.922085 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:47.922088 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.922578 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.922579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:47.922638 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:47.923027 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:41:47.923033 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:41:47.924439 (+ 1406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:41:47.926598 (+ 2159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:47.926599 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:47.926600 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:47.926828 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:47.927063 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:47.927063 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:47.927064 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:47.927192 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:47.927554 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:41:47.927556 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:41:47.927607 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 16:41:47.927660 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 16:41:47.927770 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260812 16:41:47.927857 10644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07a261b7f5014d29b67dd6c3f0b7176d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df35c5527cf545049882c590161af7a1" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36593 } } peers { permanent_uuid: "2cf47406684346a39e4733f2aa758126" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 36563 } } peers { permanent_uuid: "07a261b7f5014d29b67dd6c3f0b7176d" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 39587 } } }
I20260812 16:41:47.927953 10644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07a261b7f5014d29b67dd6c3f0b7176d [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:41:47.928045 10627 sys_catalog.cc:564] T 00000000000000000000000000000000 P 07a261b7f5014d29b67dd6c3f0b7176d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:41:47.928233 10640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:47.919227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54036 (local address 127.0.253.252:39587)
0812 16:41:47.919429 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:47.919430 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:47.919443 (+ 13us) server_negotiation.cc:407] Connection header received
0812 16:41:47.919462 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:47.919462 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:47.919473 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:47.919596 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:47.919900 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.921229 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.922180 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.922476 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.922792 (+ 316us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:47.924579 (+ 1787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:47.924580 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:47.924581 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:47.924589 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:47.926492 (+ 1903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:47.926918 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:47.926919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:47.926919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:47.926977 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:47.927271 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:47.927272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:47.927272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:47.927458 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:47.927988 (+ 530us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:47.928032 (+ 44us) server_negotiation.cc:299] Negotiation successful
0812 16:41:47.928129 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":111,"threads_started":1}
I20260812 16:41:47.929600 10361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df35c5527cf545049882c590161af7a1: Bootstrap starting.
W20260812 16:41:47.930088 10666 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 07a261b7f5014d29b67dd6c3f0b7176d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 16:41:47.930186 10666 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:41:47.930433 10361 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df35c5527cf545049882c590161af7a1: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:47.930774 10361 log.cc:824] T 00000000000000000000000000000000 P df35c5527cf545049882c590161af7a1: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:47.931871 10361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df35c5527cf545049882c590161af7a1: No bootstrap required, opened a new log
I20260812 16:41:47.933287 10361 raft_consensus.cc:348] T 00000000000000000000000000000000 P df35c5527cf545049882c590161af7a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df35c5527cf545049882c590161af7a1" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36593 } } peers { permanent_uuid: "2cf47406684346a39e4733f2aa758126" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 36563 } } peers { permanent_uuid: "07a261b7f5014d29b67dd6c3f0b7176d" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 39587 } }
I20260812 16:41:47.933427 10361 raft_consensus.cc:729] T 00000000000000000000000000000000 P df35c5527cf545049882c590161af7a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df35c5527cf545049882c590161af7a1, State: Initialized, Role: FOLLOWER
I20260812 16:41:47.933513 10361 consensus_queue.cc:260] T 00000000000000000000000000000000 P df35c5527cf545049882c590161af7a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df35c5527cf545049882c590161af7a1" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36593 } } peers { permanent_uuid: "2cf47406684346a39e4733f2aa758126" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 36563 } } peers { permanent_uuid: "07a261b7f5014d29b67dd6c3f0b7176d" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 39587 } }
I20260812 16:41:47.931511 10514 master_runner.cc:186] Error getting master registration for 127.0.253.254:36593: OK, instance_id { permanent_uuid: "df35c5527cf545049882c590161af7a1" instance_seqno: 1786552907573517 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 16:41:47.933828 10361 sys_catalog.cc:564] T 00000000000000000000000000000000 P df35c5527cf545049882c590161af7a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:41:47.933845 10315 raft_consensus.cc:482] T 00000000000000000000000000000000 P df35c5527cf545049882c590161af7a1 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 16:41:47.933954 10514 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 16:41:47.933954 10315 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df35c5527cf545049882c590161af7a1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:47.934026 10669 sys_catalog.cc:455] T 00000000000000000000000000000000 P df35c5527cf545049882c590161af7a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df35c5527cf545049882c590161af7a1" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36593 } } peers { permanent_uuid: "2cf47406684346a39e4733f2aa758126" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 36563 } } peers { permanent_uuid: "07a261b7f5014d29b67dd6c3f0b7176d" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 39587 } } }
I20260812 16:41:47.934147 10669 sys_catalog.cc:458] T 00000000000000000000000000000000 P df35c5527cf545049882c590161af7a1 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:41:47.934772 10315 raft_consensus.cc:504] T 00000000000000000000000000000000 P df35c5527cf545049882c590161af7a1 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df35c5527cf545049882c590161af7a1" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36593 } } peers { permanent_uuid: "2cf47406684346a39e4733f2aa758126" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 36563 } } peers { permanent_uuid: "07a261b7f5014d29b67dd6c3f0b7176d" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 39587 } }
I20260812 16:41:47.935030 10315 leader_election.cc:288] T 00000000000000000000000000000000 P df35c5527cf545049882c590161af7a1 [CANDIDATE]: Term 1 election: Requested vote from peers 2cf47406684346a39e4733f2aa758126 (127.0.253.253:36563), 07a261b7f5014d29b67dd6c3f0b7176d (127.0.253.252:39587)
I20260812 16:41:47.935389 10590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "df35c5527cf545049882c590161af7a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "07a261b7f5014d29b67dd6c3f0b7176d"
I20260812 16:41:47.935503 10590 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 07a261b7f5014d29b67dd6c3f0b7176d [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:47.936383 10590 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 07a261b7f5014d29b67dd6c3f0b7176d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df35c5527cf545049882c590161af7a1 in term 1.
I20260812 16:41:47.936616 10279 leader_election.cc:302] T 00000000000000000000000000000000 P df35c5527cf545049882c590161af7a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07a261b7f5014d29b67dd6c3f0b7176d, df35c5527cf545049882c590161af7a1; no voters:
I20260812 16:41:47.936877 10669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df35c5527cf545049882c590161af7a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:41:47.937027 10669 raft_consensus.cc:686] T 00000000000000000000000000000000 P df35c5527cf545049882c590161af7a1 [term 1 LEADER]: Becoming Leader. State: Replica: df35c5527cf545049882c590161af7a1, State: Running, Role: LEADER
I20260812 16:41:47.937132 10669 consensus_queue.cc:237] T 00000000000000000000000000000000 P df35c5527cf545049882c590161af7a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df35c5527cf545049882c590161af7a1" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36593 } } peers { permanent_uuid: "2cf47406684346a39e4733f2aa758126" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 36563 } } peers { permanent_uuid: "07a261b7f5014d29b67dd6c3f0b7176d" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 39587 } }
I20260812 16:41:47.937662 10678 sys_catalog.cc:455] T 00000000000000000000000000000000 P df35c5527cf545049882c590161af7a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df35c5527cf545049882c590161af7a1. Latest consensus state: current_term: 1 leader_uuid: "df35c5527cf545049882c590161af7a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df35c5527cf545049882c590161af7a1" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36593 } } peers { permanent_uuid: "2cf47406684346a39e4733f2aa758126" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 36563 } } peers { permanent_uuid: "07a261b7f5014d29b67dd6c3f0b7176d" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 39587 } } }
I20260812 16:41:47.937935 10678 sys_catalog.cc:458] T 00000000000000000000000000000000 P df35c5527cf545049882c590161af7a1 [sys.catalog]: This master's current role is: LEADER
W20260812 16:41:47.940327 10699 catalog_manager.cc:1558] T 00000000000000000000000000000000 P df35c5527cf545049882c590161af7a1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 16:41:47.940433 10699 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:41:47.940653 10685 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:41:47.941720 10685 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:41:47.940259 10246 master_runner.cc:186] Error getting master registration for 127.0.253.253:36563: OK, instance_id { permanent_uuid: "2cf47406684346a39e4733f2aa758126" instance_seqno: 1786552907722701 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 16:41:47.942924 10246 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 16:41:47.944288 10590 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 07a261b7f5014d29b67dd6c3f0b7176d [term 1 FOLLOWER]: Refusing update from remote peer df35c5527cf545049882c590161af7a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:41:47.944644 10678 consensus_queue.cc:1035] T 00000000000000000000000000000000 P df35c5527cf545049882c590161af7a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07a261b7f5014d29b67dd6c3f0b7176d" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 39587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:41:47.945539 10365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:47.936484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59290 (local address 127.0.253.254:36593)
0812 16:41:47.936541 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:47.936543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:47.936723 (+ 180us) server_negotiation.cc:407] Connection header received
0812 16:41:47.936910 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:47.936911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:47.936923 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:47.936953 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:47.937944 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.938704 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.939886 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.940199 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.940245 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:47.941671 (+ 1426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:47.941672 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:47.941673 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:47.941680 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:47.943759 (+ 2079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:47.944175 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:47.944176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:47.944177 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:47.944236 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:47.944488 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:47.944488 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:47.944489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:47.944618 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:47.944722 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:47.945372 (+ 650us) server_negotiation.cc:299] Negotiation successful
0812 16:41:47.945452 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 16:41:47.945958 10644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07a261b7f5014d29b67dd6c3f0b7176d [sys.catalog]: SysCatalogTable state changed. Reason: New leader df35c5527cf545049882c590161af7a1. Latest consensus state: current_term: 1 leader_uuid: "df35c5527cf545049882c590161af7a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df35c5527cf545049882c590161af7a1" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36593 } } peers { permanent_uuid: "2cf47406684346a39e4733f2aa758126" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 36563 } } peers { permanent_uuid: "07a261b7f5014d29b67dd6c3f0b7176d" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 39587 } } }
I20260812 16:41:47.946055 10644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07a261b7f5014d29b67dd6c3f0b7176d [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:41:47.948206 10678 sys_catalog.cc:455] T 00000000000000000000000000000000 P df35c5527cf545049882c590161af7a1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "df35c5527cf545049882c590161af7a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df35c5527cf545049882c590161af7a1" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36593 } } peers { permanent_uuid: "2cf47406684346a39e4733f2aa758126" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 36563 } } peers { permanent_uuid: "07a261b7f5014d29b67dd6c3f0b7176d" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 39587 } } }
I20260812 16:41:47.948309 10678 sys_catalog.cc:458] T 00000000000000000000000000000000 P df35c5527cf545049882c590161af7a1 [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:47.948565 10704 mvcc.cc:205] Tried to move back new op lower bound from 7317720710936395776 to 7317720710911320064. Current Snapshot: MvccSnapshot[applied={T|T < 7317720710936395776}]
I20260812 16:41:47.948725 10644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07a261b7f5014d29b67dd6c3f0b7176d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "df35c5527cf545049882c590161af7a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df35c5527cf545049882c590161af7a1" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36593 } } peers { permanent_uuid: "2cf47406684346a39e4733f2aa758126" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 36563 } } peers { permanent_uuid: "07a261b7f5014d29b67dd6c3f0b7176d" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 39587 } } }
I20260812 16:41:47.948845 10644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07a261b7f5014d29b67dd6c3f0b7176d [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:41:47.951807 10685 catalog_manager.cc:1347] Generated new cluster ID: 4bf326d40d7d40fb8539adfbc50db251
I20260812 16:41:47.951855 10685 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:41:47.953577 10504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:47.936809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43140 (local address 127.0.253.253:36563)
0812 16:41:47.936852 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:47.936854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:47.943237 (+ 6383us) server_negotiation.cc:407] Connection header received
0812 16:41:47.943353 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:47.943354 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:47.943366 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:47.943397 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:47.944082 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.944762 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.947555 (+ 2793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.948507 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.948556 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:47.950346 (+ 1790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:47.950348 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:47.950348 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:47.950356 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:47.952206 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:47.952691 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:47.952692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:47.952693 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:47.952755 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:47.952963 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:47.952964 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:47.952964 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:47.953095 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:47.953166 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:47.953388 (+ 222us) server_negotiation.cc:299] Negotiation successful
0812 16:41:47.953475 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 16:41:47.955370 10640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:47.939829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54052 (local address 127.0.253.252:39587)
0812 16:41:47.939869 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:47.939871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:47.946123 (+ 6252us) server_negotiation.cc:407] Connection header received
0812 16:41:47.946146 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:47.946146 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:47.946157 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:47.946217 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:47.946896 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.947646 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.949877 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.950234 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.950293 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:47.952041 (+ 1748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:47.952042 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:47.952042 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:47.952049 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:47.953749 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:47.954209 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:47.954210 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:47.954210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:47.954275 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:47.954485 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:47.954485 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:47.954486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:47.954614 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:47.954689 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:47.955206 (+ 517us) server_negotiation.cc:299] Negotiation successful
0812 16:41:47.955282 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 16:41:47.955888 10703 mvcc.cc:205] Tried to move back new op lower bound from 7317720710936395776 to 7317720710911320064. Current Snapshot: MvccSnapshot[applied={T|T < 7317720710936395776 or (T in {7317720710936395776})}]
I20260812 16:41:47.960189 10493 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.253.252" port: 39587 } attempt: 3
I20260812 16:41:47.963860 10685 catalog_manager.cc:1370] Generated new certificate authority record
Aug 12 16:41:47 dist-test-slave-w4v5 krb5kdc[10224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552907, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.253@KRBTEST.COM for kudu/127.0.253.252@KRBTEST.COM
I20260812 16:41:47.965232 10685 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:41:47.968492 10500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:47.960457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:39587 (local address 127.0.0.1:54062)
0812 16:41:47.960497 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:47.960510 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:47.960554 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:47.960774 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:47.960776 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:47.960968 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:41:47.961021 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:47.961023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.962041 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.962042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:47.962758 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:47.962761 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.963386 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.963387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:47.963446 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:47.963922 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:41:47.963928 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:41:47.965530 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:41:47.967392 (+ 1862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:47.967394 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:47.967394 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:47.967592 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:47.967792 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:47.967793 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:47.967793 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:47.967857 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:47.968152 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:41:47.968153 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:41:47.968205 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 16:41:47.968271 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 16:41:47.968378 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260812 16:41:47.968881 10640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:47.960610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54062 (local address 127.0.253.252:39587)
0812 16:41:47.960649 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:47.960651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:47.960660 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:41:47.960681 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:47.960682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:47.960692 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:47.961095 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:47.961131 (+ 36us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.961940 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.962851 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:47.963289 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:47.963627 (+ 338us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:47.965605 (+ 1978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:47.965607 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:47.965607 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:47.965615 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:47.967322 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:47.967657 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:47.967659 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:47.967659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:47.967718 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:47.967924 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:47.967925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:47.967925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:47.968060 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:47.968121 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:47.968698 (+ 577us) server_negotiation.cc:299] Negotiation successful
0812 16:41:47.968784 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 16:41:47.970471 10493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cf47406684346a39e4733f2aa758126: Bootstrap starting.
I20260812 16:41:47.971287 10493 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2cf47406684346a39e4733f2aa758126: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:47.971586 10493 log.cc:824] T 00000000000000000000000000000000 P 2cf47406684346a39e4733f2aa758126: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:47.976178 10493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cf47406684346a39e4733f2aa758126: No bootstrap required, opened a new log
I20260812 16:41:47.978096 10493 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2cf47406684346a39e4733f2aa758126 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df35c5527cf545049882c590161af7a1" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36593 } } peers { permanent_uuid: "2cf47406684346a39e4733f2aa758126" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 36563 } } peers { permanent_uuid: "07a261b7f5014d29b67dd6c3f0b7176d" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 39587 } }
I20260812 16:41:47.978240 10493 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2cf47406684346a39e4733f2aa758126 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cf47406684346a39e4733f2aa758126, State: Initialized, Role: FOLLOWER
I20260812 16:41:47.978322 10493 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2cf47406684346a39e4733f2aa758126 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df35c5527cf545049882c590161af7a1" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36593 } } peers { permanent_uuid: "2cf47406684346a39e4733f2aa758126" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 36563 } } peers { permanent_uuid: "07a261b7f5014d29b67dd6c3f0b7176d" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 39587 } }
I20260812 16:41:47.978662 10493 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2cf47406684346a39e4733f2aa758126 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:41:47.979169 10446 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2cf47406684346a39e4733f2aa758126 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:47.980175 10446 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2cf47406684346a39e4733f2aa758126 [term 1 FOLLOWER]: Refusing update from remote peer df35c5527cf545049882c590161af7a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:41:47.980177 10685 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df35c5527cf545049882c590161af7a1: Generated new TSK 0
I20260812 16:41:47.980346 10685 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:41:47.980371 10449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "df35c5527cf545049882c590161af7a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2cf47406684346a39e4733f2aa758126"
I20260812 16:41:47.980967 10449 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2cf47406684346a39e4733f2aa758126 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df35c5527cf545049882c590161af7a1 in term 1.
I20260812 16:41:47.981353 10669 consensus_queue.cc:1035] T 00000000000000000000000000000000 P df35c5527cf545049882c590161af7a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cf47406684346a39e4733f2aa758126" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 36563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:41:47.983307 10711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cf47406684346a39e4733f2aa758126 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df35c5527cf545049882c590161af7a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df35c5527cf545049882c590161af7a1" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36593 } } peers { permanent_uuid: "2cf47406684346a39e4733f2aa758126" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 36563 } } peers { permanent_uuid: "07a261b7f5014d29b67dd6c3f0b7176d" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 39587 } } }
I20260812 16:41:47.983441 10711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cf47406684346a39e4733f2aa758126 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:41:47.989444 10711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cf47406684346a39e4733f2aa758126 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "df35c5527cf545049882c590161af7a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df35c5527cf545049882c590161af7a1" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36593 } } peers { permanent_uuid: "2cf47406684346a39e4733f2aa758126" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 36563 } } peers { permanent_uuid: "07a261b7f5014d29b67dd6c3f0b7176d" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 39587 } } }
I20260812 16:41:47.989810 10711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cf47406684346a39e4733f2aa758126 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:41:47.990610 10736 mvcc.cc:205] Tried to move back new op lower bound from 7317720711080243200 to 7317720710911320064. Current Snapshot: MvccSnapshot[applied={T|T < 7317720710936395776}]
I20260812 16:41:47.990869 10726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cf47406684346a39e4733f2aa758126 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df35c5527cf545049882c590161af7a1. Latest consensus state: current_term: 1 leader_uuid: "df35c5527cf545049882c590161af7a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df35c5527cf545049882c590161af7a1" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36593 } } peers { permanent_uuid: "2cf47406684346a39e4733f2aa758126" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 36563 } } peers { permanent_uuid: "07a261b7f5014d29b67dd6c3f0b7176d" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 39587 } } }
I20260812 16:41:47.990955 10726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cf47406684346a39e4733f2aa758126 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:41:47.993026 10743 catalog_manager.cc:1259] Loaded cluster ID: 4bf326d40d7d40fb8539adfbc50db251
I20260812 16:41:47.993083 10743 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2cf47406684346a39e4733f2aa758126: loading cluster ID for follower catalog manager: success
I20260812 16:41:47.994213 10743 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2cf47406684346a39e4733f2aa758126: acquiring CA information for follower catalog manager: success
I20260812 16:41:47.994545 10743 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2cf47406684346a39e4733f2aa758126: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 16:41:48.014967 10365 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:41:48.013245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59296 (local address 127.0.253.254:36593)
0812 16:41:48.013295 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:48.013298 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:48.014260 (+ 962us) server_negotiation.cc:407] Connection header received
0812 16:41:48.014371 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:48.014372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:48.014384 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:48.014415 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:48.014876 (+ 461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59296: BlockingRecv error: recv got EOF from 127.0.0.1:59296 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260812 16:41:48.015789 10504 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:41:48.013614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43154 (local address 127.0.253.253:36563)
0812 16:41:48.013659 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:48.013661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:48.015280 (+ 1619us) server_negotiation.cc:407] Connection header received
0812 16:41:48.015379 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:48.015380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:48.015391 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:48.015420 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:48.015725 (+ 305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43154: BlockingRecv error: recv got EOF from 127.0.0.1:43154 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260812 16:41:48.016636 10640 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:41:48.013966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54072 (local address 127.0.253.252:39587)
0812 16:41:48.014007 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:48.014009 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:48.015999 (+ 1990us) server_negotiation.cc:407] Connection header received
0812 16:41:48.016180 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:48.016181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:48.016195 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:48.016225 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:48.016571 (+ 346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54072: BlockingRecv error: recv got EOF from 127.0.0.1:54072 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 16:41:48.021833 10504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:48.018269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43164 (local address 127.0.253.253:36563)
0812 16:41:48.018306 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:48.018308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:48.018514 (+ 206us) server_negotiation.cc:407] Connection header received
0812 16:41:48.018622 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:48.018623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:48.018672 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:48.018723 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:41:48.018915 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:48.019759 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:48.020840 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:48.021153 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:48.021193 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:48.021419 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:41:48.021566 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:41:48.021621 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:48.021741 (+ 120us) server_negotiation.cc:299] Negotiation successful
0812 16:41:48.021760 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260812 16:41:48.025231 10640 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:41:48.024390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54074 (local address 127.0.253.252:39587)
0812 16:41:48.024430 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:48.024432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:48.024626 (+ 194us) server_negotiation.cc:407] Connection header received
0812 16:41:48.024743 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:48.024744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:48.024765 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:48.024795 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:48.025174 (+ 379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54074: BlockingRecv error: recv got EOF from 127.0.0.1:54074 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 16:41:48.031010 10504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:48.027551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43168 (local address 127.0.253.253:36563)
0812 16:41:48.027590 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:48.027592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:48.027887 (+ 295us) server_negotiation.cc:407] Connection header received
0812 16:41:48.027983 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:48.027984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:48.027994 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:48.028024 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:41:48.028208 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:48.029010 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:48.030199 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:48.030516 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:48.030557 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:48.030753 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:41:48.030786 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:41:48.030813 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:48.030915 (+ 102us) server_negotiation.cc:299] Negotiation successful
0812 16:41:48.030934 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 16:41:48.034502 10640 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:41:48.033705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54088 (local address 127.0.253.252:39587)
0812 16:41:48.033744 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:48.033746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:48.033941 (+ 195us) server_negotiation.cc:407] Connection header received
0812 16:41:48.034038 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:48.034039 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:48.034050 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:48.034078 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:48.034445 (+ 367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54088: BlockingRecv error: recv got EOF from 127.0.0.1:54088 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 16:41:48.040956 10504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:48.037662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43180 (local address 127.0.253.253:36563)
0812 16:41:48.037701 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:48.037703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:48.037973 (+ 270us) server_negotiation.cc:407] Connection header received
0812 16:41:48.038062 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:48.038063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:48.038073 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:48.038112 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:41:48.038309 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:48.039064 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:48.040169 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:48.040457 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:48.040498 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:48.040690 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:41:48.040726 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:41:48.040753 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:48.040856 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 16:41:48.040878 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260812 16:41:48.044123 10640 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:41:48.043305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54102 (local address 127.0.253.252:39587)
0812 16:41:48.043343 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:48.043345 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:48.043552 (+ 207us) server_negotiation.cc:407] Connection header received
0812 16:41:48.043653 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:48.043655 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:48.043665 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:48.043694 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:48.044057 (+ 363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54102: BlockingRecv error: recv got EOF from 127.0.0.1:54102 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 16:41:48.052655 10504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:48.049331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43182 (local address 127.0.253.253:36563)
0812 16:41:48.049370 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:48.049373 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:48.049634 (+ 261us) server_negotiation.cc:407] Connection header received
0812 16:41:48.049736 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:48.049737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:48.049747 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:48.049774 (+ 27us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:41:48.049946 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:48.050725 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:48.051812 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:48.052127 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:48.052172 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:48.052397 (+ 225us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:41:48.052432 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:41:48.052458 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:48.052559 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 16:41:48.052580 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 16:41:48.055897 10640 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:41:48.055017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54112 (local address 127.0.253.252:39587)
0812 16:41:48.055058 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:48.055060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:48.055330 (+ 270us) server_negotiation.cc:407] Connection header received
0812 16:41:48.055434 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:48.055435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:48.055448 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:48.055479 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:48.055841 (+ 362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54112: BlockingRecv error: recv got EOF from 127.0.0.1:54112 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 16:41:48.068783 10504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:48.065242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43190 (local address 127.0.253.253:36563)
0812 16:41:48.065288 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:48.065290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:48.065600 (+ 310us) server_negotiation.cc:407] Connection header received
0812 16:41:48.065711 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:48.065712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:48.065723 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:48.065754 (+ 31us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:41:48.065951 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:48.066681 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:48.067962 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:48.068250 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:48.068299 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:48.068507 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:41:48.068542 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:41:48.068571 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:48.068682 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 16:41:48.068702 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 16:41:48.072580 10640 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:41:48.071637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54122 (local address 127.0.253.252:39587)
0812 16:41:48.071695 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:48.071697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:48.071971 (+ 274us) server_negotiation.cc:407] Connection header received
0812 16:41:48.072073 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:48.072074 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:48.072096 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:48.072127 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:48.072509 (+ 382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54122: BlockingRecv error: recv got EOF from 127.0.0.1:54122 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 16:41:48.093571 10504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:48.089999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43202 (local address 127.0.253.253:36563)
0812 16:41:48.090054 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:48.090057 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:48.090388 (+ 331us) server_negotiation.cc:407] Connection header received
0812 16:41:48.090532 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:48.090534 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:48.090548 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:48.090585 (+ 37us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:41:48.090788 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:48.091543 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:48.092735 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:48.093014 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:48.093058 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:48.093291 (+ 233us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:41:48.093331 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:41:48.093360 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:48.093469 (+ 109us) server_negotiation.cc:299] Negotiation successful
0812 16:41:48.093491 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260812 16:41:48.097007 10640 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:41:48.096074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54138 (local address 127.0.253.252:39587)
0812 16:41:48.096133 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:48.096135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:48.096422 (+ 287us) server_negotiation.cc:407] Connection header received
0812 16:41:48.096524 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:48.096525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:48.096536 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:48.096565 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:48.096951 (+ 386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54138: BlockingRecv error: recv got EOF from 127.0.0.1:54138 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 16:41:48.133176 10504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:48.130404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43218 (local address 127.0.253.253:36563)
0812 16:41:48.130460 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:48.130462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:48.130627 (+ 165us) server_negotiation.cc:407] Connection header received
0812 16:41:48.130708 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:48.130710 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:48.130722 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:48.130802 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:41:48.130969 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:48.131522 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:48.132426 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:48.132628 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:48.132712 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:48.132885 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:41:48.132927 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:41:48.132991 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:48.133057 (+ 66us) server_negotiation.cc:299] Negotiation successful
0812 16:41:48.133080 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260812 16:41:48.135751 10640 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:41:48.135140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54146 (local address 127.0.253.252:39587)
0812 16:41:48.135195 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:48.135197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:48.135205 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:41:48.135230 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:48.135230 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:48.135241 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:48.135359 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:48.135696 (+ 337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54146: BlockingRecv error: recv got EOF from 127.0.0.1:54146 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 16:41:48.203621 10504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:48.201343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43222 (local address 127.0.253.253:36563)
0812 16:41:48.201424 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:48.201426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:48.201522 (+ 96us) server_negotiation.cc:407] Connection header received
0812 16:41:48.201572 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:48.201573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:48.201582 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:48.201651 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:41:48.201807 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:48.202307 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:48.203050 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:48.203252 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:48.203303 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:48.203431 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:41:48.203458 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:41:48.203496 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:48.203545 (+ 49us) server_negotiation.cc:299] Negotiation successful
0812 16:41:48.203562 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260812 16:41:48.206139 10640 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:41:48.205454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54156 (local address 127.0.253.252:39587)
0812 16:41:48.205530 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:48.205532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:48.205557 (+ 25us) server_negotiation.cc:407] Connection header received
0812 16:41:48.205626 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:48.205627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:48.205641 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:48.205686 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:48.206083 (+ 397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54156: BlockingRecv error: recv got EOF from 127.0.0.1:54156 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 16:41:48.338642 10504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:48.335667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43238 (local address 127.0.253.253:36563)
0812 16:41:48.335752 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:48.335755 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:48.335879 (+ 124us) server_negotiation.cc:407] Connection header received
0812 16:41:48.335986 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:48.335987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:48.336000 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:48.336098 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:41:48.336246 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:48.336747 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:48.337722 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:48.337917 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:48.338017 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:48.338175 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:41:48.338206 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:41:48.338271 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:48.338436 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 16:41:48.338458 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260812 16:41:48.341120 10640 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:41:48.340545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54166 (local address 127.0.253.252:39587)
0812 16:41:48.340588 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:48.340590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:48.340636 (+ 46us) server_negotiation.cc:407] Connection header received
0812 16:41:48.340691 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:48.340692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:48.340703 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:48.340753 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:48.341041 (+ 288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54166: BlockingRecv error: recv got EOF from 127.0.0.1:54166 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 16:41:48.482859 10669 sys_catalog.cc:455] T 00000000000000000000000000000000 P df35c5527cf545049882c590161af7a1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "df35c5527cf545049882c590161af7a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df35c5527cf545049882c590161af7a1" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 36593 } } peers { permanent_uuid: "2cf47406684346a39e4733f2aa758126" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 36563 } } peers { permanent_uuid: "07a261b7f5014d29b67dd6c3f0b7176d" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 39587 } } }
I20260812 16:41:48.483039 10669 sys_catalog.cc:458] T 00000000000000000000000000000000 P df35c5527cf545049882c590161af7a1 [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:48.601008 10504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:48.598539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43242 (local address 127.0.253.253:36563)
0812 16:41:48.598626 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:48.598628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:48.598639 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:41:48.598733 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:48.598734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:48.598747 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:48.598823 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:41:48.598969 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:48.599487 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:48.600302 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:48.600495 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:48.600572 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:48.600734 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:41:48.600772 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:41:48.600833 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:48.600890 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 16:41:48.600912 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260812 16:41:48.603441 10640 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:41:48.602780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54180 (local address 127.0.253.252:39587)
0812 16:41:48.602843 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:48.602845 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:48.602855 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:41:48.602927 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:48.602928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:48.602940 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:48.603013 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:48.603378 (+ 365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54180: BlockingRecv error: recv got EOF from 127.0.0.1:54180 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 16:41:48.930729 10666 catalog_manager.cc:1259] Loaded cluster ID: 4bf326d40d7d40fb8539adfbc50db251
I20260812 16:41:48.930817 10666 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 07a261b7f5014d29b67dd6c3f0b7176d: loading cluster ID for follower catalog manager: success
I20260812 16:41:48.931687 10666 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 07a261b7f5014d29b67dd6c3f0b7176d: acquiring CA information for follower catalog manager: success
I20260812 16:41:48.931968 10666 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 07a261b7f5014d29b67dd6c3f0b7176d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 16:41:49.119035 10966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:49.116638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43256 (local address 127.0.253.253:36563)
0812 16:41:49.116803 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:49.116805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:49.116862 (+ 57us) server_negotiation.cc:407] Connection header received
0812 16:41:49.116916 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:49.116917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:49.116928 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:49.117004 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:41:49.117172 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:49.117675 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:49.118428 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:49.118624 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:49.118682 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:49.118829 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:41:49.118857 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:41:49.118897 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:49.118947 (+ 50us) server_negotiation.cc:299] Negotiation successful
0812 16:41:49.118966 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":107,"threads_started":1}
I20260812 16:41:49.123147 10976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:49.120785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54196 (local address 127.0.253.252:39587)
0812 16:41:49.120895 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:49.120897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:49.120904 (+ 7us) server_negotiation.cc:407] Connection header received
0812 16:41:49.120926 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:49.120927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:49.120981 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:49.121052 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:41:49.121180 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:49.121645 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:49.122335 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:49.122526 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:49.122581 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:49.122766 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:41:49.122876 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:41:49.122936 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:49.122988 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 16:41:49.123006 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":64,"threads_started":1}
I20260812 16:41:49.127884 10966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:49.125140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43264 (local address 127.0.253.253:36563)
0812 16:41:49.125180 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:49.125182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:49.125217 (+ 35us) server_negotiation.cc:407] Connection header received
0812 16:41:49.125326 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:49.125327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:49.125338 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:49.125386 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:41:49.125540 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:49.126000 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:49.127027 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:49.127332 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:49.127402 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:49.127596 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:41:49.127629 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:41:49.127681 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:49.127791 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 16:41:49.127818 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 16:41:49.128328 10988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:49.124866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59304 (local address 127.0.253.254:36593)
0812 16:41:49.125118 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:49.125120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:49.125128 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:41:49.125228 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:49.125229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:49.125301 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:49.125384 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:41:49.125492 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:49.126244 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:49.127260 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:49.127615 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:49.127689 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:49.127886 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:41:49.127995 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:41:49.128049 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:49.128225 (+ 176us) server_negotiation.cc:299] Negotiation successful
0812 16:41:49.128248 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":56,"threads_started":1}
I20260812 16:41:49.129118 10976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:49.124841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54210 (local address 127.0.253.252:39587)
0812 16:41:49.124892 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:49.124895 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:49.125223 (+ 328us) server_negotiation.cc:407] Connection header received
0812 16:41:49.125272 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:49.125273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:49.125282 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:49.125338 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:41:49.125513 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:49.126234 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:49.128158 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:49.128522 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:49.128780 (+ 258us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:49.128868 (+ 88us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:41:49.128901 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:41:49.128945 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:49.129022 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 16:41:49.129041 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 16:41:49.130355 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 10246
I20260812 16:41:49.147960 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 10369
I20260812 16:41:49.161157 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 10514
2026-08-12T16:41:49Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3199 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:41:49 dist-test-slave-w4v5 krb5kdc[10994](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:41:49 dist-test-slave-w4v5 krb5kdc[10994](info): set up 2 sockets
Aug 12 16:41:49 dist-test-slave-w4v5 krb5kdc[10994](info): commencing operation
krb5kdc: starting...
W20260812 16:41:50.486802 1015 mini_kdc.cc:121] Time spent starting KDC: real 1.298s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/krb5kdc/test-user.keytab.
Aug 12 16:41:50 dist-test-slave-w4v5 krb5kdc[10994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:41:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:41:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
Entry for principal kudu/127.0.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab.
I20260812 16:41:50.637637 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:34605
--webserver_interface=127.0.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:40577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:34605,127.0.253.253:45221,127.0.253.252:38839
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:41:50.721016 11022 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 16:41:50.721282 11022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:50.721312 11022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:50.721354 11022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:50.722515 11022 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:41:50.722582 11022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:50.722607 11022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:50.722630 11022 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:41:50.722659 11022 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:41:50.724241 11022 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:40577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:34605,127.0.253.253:45221,127.0.253.252:38839
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.254.keytab
--principal=kudu/127.0.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.254:34605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:50.724542 11022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:50.724777 11022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:50.729983 11036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:50.730007 11032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:50.730154 11022 server_base.cc:1034] running on GCE node
W20260812 16:41:50.729987 11031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:50.730566 11022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:50.730919 11022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:50.732084 11022 hybrid_clock.cc:648] HybridClock initialized: now 1786552910732059 us; error 43 us; skew 500 ppm
Aug 12 16:41:50 dist-test-slave-w4v5 krb5kdc[10994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552910, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:41:50.734637 11022 init.cc:377] Logged in from keytab as kudu/127.0.253.254@KRBTEST.COM (short username kudu)
I20260812 16:41:50.735142 11022 webserver.cc:460] Webserver started at http://127.0.253.254:34229/ using document root <none> and password file <none>
I20260812 16:41:50.735389 11022 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:50.735459 11022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:50.735597 11022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:50.736706 11022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-0/data/instance:
uuid: "24a1302bbbeb4ad59f83a6cc9e812209"
format_stamp: "Formatted at 2026-08-12 16:41:50 on dist-test-slave-w4v5"
server_key: "ec08dfa3ab7fd6062de300d7ab4e78c9"
server_key_iv: "fc6921ac0698a53ab7dc1fcdaf235515"
server_key_version: "encryptionkey@0"
I20260812 16:41:50.737071 11022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-0/wal/instance:
uuid: "24a1302bbbeb4ad59f83a6cc9e812209"
format_stamp: "Formatted at 2026-08-12 16:41:50 on dist-test-slave-w4v5"
server_key: "ec08dfa3ab7fd6062de300d7ab4e78c9"
server_key_iv: "fc6921ac0698a53ab7dc1fcdaf235515"
server_key_version: "encryptionkey@0"
I20260812 16:41:50.738457 11022 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:41:50.739253 11047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:50.739421 11022 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:41:50.739518 11022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-0/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-0/wal
uuid: "24a1302bbbeb4ad59f83a6cc9e812209"
format_stamp: "Formatted at 2026-08-12 16:41:50 on dist-test-slave-w4v5"
server_key: "ec08dfa3ab7fd6062de300d7ab4e78c9"
server_key_iv: "fc6921ac0698a53ab7dc1fcdaf235515"
server_key_version: "encryptionkey@0"
I20260812 16:41:50.739601 11022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:50.762878 11022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:50.765856 11022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:50.766010 11022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:50.770614 11022 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.254:34605
I20260812 16:41:50.770651 11130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.254:34605 every 8 connection(s)
I20260812 16:41:50.771106 11022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-0/data/info.pb
I20260812 16:41:50.772217 11131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:50.773476 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 11022
I20260812 16:41:50.773572 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-0/wal/instance
I20260812 16:41:50.773783 1015 external_mini_cluster.cc:1428] Setting key c622f5898155fc2c07c92afd816452e3
I20260812 16:41:50.774194 11131 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.253.254" port: 34605 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 16:41:50 dist-test-slave-w4v5 krb5kdc[10994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552910, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.253.253@KRBTEST.COM; defaulting to no policy
I20260812 16:41:50.784616 11134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:50.775035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:34605 (local address 127.0.0.1:58944)
0812 16:41:50.775289 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:50.775330 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:50.775810 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:50.776123 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:50.776125 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:50.776417 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:41:50.776573 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:50.776578 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:50.777649 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:50.777650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:50.778355 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:50.778357 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:50.778894 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:50.778896 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:50.778963 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:50.779594 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:41:50.779604 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:41:50.780973 (+ 1369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:41:50.783013 (+ 2040us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:50.783020 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:50.783022 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:50.783279 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:50.783539 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:50.783540 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:50.783540 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:50.783637 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:50.784000 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:41:50.784005 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:41:50.784069 (+ 64us) client_negotiation.cc:769] Sending connection context
0812 16:41:50.784175 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 16:41:50.784427 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":106,"thread_start_us":169,"threads_started":1}
I20260812 16:41:50.784614 11135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:50.775114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58944 (local address 127.0.253.254:34605)
0812 16:41:50.775312 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:50.775323 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:50.775405 (+ 82us) server_negotiation.cc:407] Connection header received
0812 16:41:50.775994 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:50.775996 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:50.776021 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:50.776119 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:50.776683 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:50.777549 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:50.778455 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:50.778773 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:50.778886 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:50.781102 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:50.781110 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:50.781110 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:50.781117 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:50.782910 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:50.783379 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:50.783382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:50.783383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:50.783468 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:50.783711 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:50.783712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:50.783713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:50.783926 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:50.784021 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:50.784227 (+ 206us) server_negotiation.cc:299] Negotiation successful
0812 16:41:50.784362 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":105,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.253@KRBTEST.COM" created.
I20260812 16:41:50.785468 11131 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.253.253" port: 45221 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:41:50.785864 11134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:50.785721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.253:45221 (local address 127.0.0.1:40942)
0812 16:41:50.785777 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:50.785832 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.253:45221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 16:41:50.786064 11053 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.253.253:45221: connect: Connection refused (error 111)
I20260812 16:41:50.786561 11134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:50.786446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.253:45221 (local address 127.0.0.1:40956)
0812 16:41:50.786497 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:50.786529 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.253:45221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 16:41:50.786715 11131 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.253.253:45221: Network error: Client connection negotiation failed: client connection to 127.0.253.253:45221: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.253.keytab.
Entry for principal kudu/127.0.253.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.253.keytab.
I20260812 16:41:50.801704 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.253:45221
--webserver_interface=127.0.253.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.253.keytab
--principal=kudu/127.0.253.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:40577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:34605,127.0.253.253:45221,127.0.253.252:38839
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:41:50.836014 11131 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.253.253" port: 45221 } attempt: 1
I20260812 16:41:50.836617 11134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:50.836430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.253:45221 (local address 127.0.0.1:40958)
0812 16:41:50.836506 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:50.836585 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.253:45221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 16:41:50.837005 11134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:50.836897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.253:45221 (local address 127.0.0.1:40962)
0812 16:41:50.836948 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:50.836975 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.253:45221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260812 16:41:50.837107 11131 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.253.253:45221: Network error: Client connection negotiation failed: client connection to 127.0.253.253:45221: connect: Connection refused (error 111)
W20260812 16:41:50.888818 11144 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 16:41:50.889076 11144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:50.889119 11144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:50.889159 11144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:50.890331 11144 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:41:50.890396 11144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:50.890420 11144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:50.890452 11144 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:41:50.890482 11144 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:41:50.892175 11144 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:40577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:34605,127.0.253.253:45221,127.0.253.252:38839
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.253.keytab
--principal=kudu/127.0.253.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.253:45221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.253.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11144
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:50.892460 11144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:50.892699 11144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:50.897742 11157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:50.897754 11163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:50.897749 11155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:50.898025 11144 server_base.cc:1034] running on GCE node
I20260812 16:41:50.898298 11144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:50.898551 11144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:50.899712 11144 hybrid_clock.cc:648] HybridClock initialized: now 1786552910899683 us; error 28 us; skew 500 ppm
Aug 12 16:41:50 dist-test-slave-w4v5 krb5kdc[10994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552910, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:41:50.902287 11144 init.cc:377] Logged in from keytab as kudu/127.0.253.253@KRBTEST.COM (short username kudu)
I20260812 16:41:50.902738 11144 webserver.cc:460] Webserver started at http://127.0.253.253:37959/ using document root <none> and password file <none>
I20260812 16:41:50.903023 11144 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:50.903110 11144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:50.903259 11144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:50.904332 11144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-1/data/instance:
uuid: "4352a30003a84cdf8700225af97e098d"
format_stamp: "Formatted at 2026-08-12 16:41:50 on dist-test-slave-w4v5"
server_key: "9d6a25bc0a623d1811ed1235c7e017be"
server_key_iv: "810cbd2810a3bda5d443d1679287e957"
server_key_version: "encryptionkey@0"
I20260812 16:41:50.904708 11144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-1/wal/instance:
uuid: "4352a30003a84cdf8700225af97e098d"
format_stamp: "Formatted at 2026-08-12 16:41:50 on dist-test-slave-w4v5"
server_key: "9d6a25bc0a623d1811ed1235c7e017be"
server_key_iv: "810cbd2810a3bda5d443d1679287e957"
server_key_version: "encryptionkey@0"
I20260812 16:41:50.905251 11131 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.253.253" port: 45221 } attempt: 2
I20260812 16:41:50.905654 11134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:50.905530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.253:45221 (local address 127.0.0.1:40970)
0812 16:41:50.905594 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:50.905626 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.253:45221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 16:41:50.906013 11134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:50.905920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.253:45221 (local address 127.0.0.1:40974)
0812 16:41:50.905966 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:50.905990 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.253:45221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 16:41:50.906105 11131 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.253.253:45221: Network error: Client connection negotiation failed: client connection to 127.0.253.253:45221: connect: Connection refused (error 111)
I20260812 16:41:50.906391 11144 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:41:50.907327 11181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:50.907632 11144 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:41:50.907763 11144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-1/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-1/wal
uuid: "4352a30003a84cdf8700225af97e098d"
format_stamp: "Formatted at 2026-08-12 16:41:50 on dist-test-slave-w4v5"
server_key: "9d6a25bc0a623d1811ed1235c7e017be"
server_key_iv: "810cbd2810a3bda5d443d1679287e957"
server_key_version: "encryptionkey@0"
I20260812 16:41:50.907841 11144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:50.926129 11144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:50.928427 11144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:50.928591 11144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:50.932819 11144 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.253:45221
I20260812 16:41:50.932879 11270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.253:45221 every 8 connection(s)
I20260812 16:41:50.933238 11144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-1/data/info.pb
I20260812 16:41:50.934316 11271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:50.936049 11271 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.253.254" port: 34605 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:41:50.937772 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 11144
I20260812 16:41:50.937860 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-1/wal/instance
I20260812 16:41:50.938059 1015 external_mini_cluster.cc:1428] Setting key b7400f96204817323bc7381fedca3d94
Aug 12 16:41:50 dist-test-slave-w4v5 krb5kdc[10994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552910, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.253@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:41:50.945717 11135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:50.936787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58950 (local address 127.0.253.254:34605)
0812 16:41:50.936869 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:50.936872 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:50.937051 (+ 179us) server_negotiation.cc:407] Connection header received
0812 16:41:50.937484 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:50.937486 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:50.937495 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:50.937547 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:50.938271 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:50.938774 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:50.939975 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:50.940203 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:50.940256 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:50.942585 (+ 2329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:50.942587 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:50.942587 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:50.942595 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:50.944248 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:50.944712 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:50.944714 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:50.944714 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:50.944782 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:50.945041 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:50.945042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:50.945042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:50.945168 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:50.945244 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:50.945527 (+ 283us) server_negotiation.cc:299] Negotiation successful
0812 16:41:50.945616 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 16:41:50.945902 11273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:50.936747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:34605 (local address 127.0.0.1:58950)
0812 16:41:50.936931 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:50.936967 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:50.937404 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:50.937600 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:50.937604 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:50.937921 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:41:50.938176 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:50.938181 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:50.938873 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:50.938874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:50.939871 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:50.939874 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:50.940304 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:50.940306 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:50.940474 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:50.941030 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:41:50.941041 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:41:50.942471 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:41:50.944334 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:50.944341 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:50.944343 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:50.944638 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:50.944851 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:50.944852 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:50.944853 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:50.944965 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:50.945247 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:41:50.945255 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:41:50.945394 (+ 139us) client_negotiation.cc:769] Sending connection context
0812 16:41:50.945505 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 16:41:50.945658 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":102,"threads_started":1}
WARNING: no policy specified for kudu/127.0.253.252@KRBTEST.COM; defaulting to no policy
I20260812 16:41:50.946775 11271 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.253.253" port: 45221 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.253.252@KRBTEST.COM" created.
Aug 12 16:41:50 dist-test-slave-w4v5 krb5kdc[10994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552910, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.253@KRBTEST.COM for kudu/127.0.253.253@KRBTEST.COM
I20260812 16:41:50.954449 11273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:50.947059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.253:45221 (local address 127.0.0.1:40980)
0812 16:41:50.947129 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:50.947140 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:50.947181 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:50.947440 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:50.947441 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:50.947582 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:41:50.947627 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:50.947628 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:50.948509 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:50.948510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:50.949135 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:50.949137 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:50.949718 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:50.949719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:50.949785 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:50.950203 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:41:50.950208 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:41:50.951510 (+ 1302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:41:50.953352 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:50.953353 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:50.953353 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:50.953514 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:50.953746 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:50.953747 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:50.953747 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:50.953807 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:50.954094 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:41:50.954095 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:41:50.954133 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 16:41:50.954236 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 16:41:50.954335 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260812 16:41:50.954540 11276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:50.947156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40980 (local address 127.0.253.253:45221)
0812 16:41:50.947312 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:50.947316 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:50.947331 (+ 15us) server_negotiation.cc:407] Connection header received
0812 16:41:50.947347 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:50.947348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:50.947359 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:50.947440 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:50.947726 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:50.948410 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:50.949218 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:50.949615 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:50.949715 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:50.951599 (+ 1884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:50.951608 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:50.951609 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:50.951617 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:50.953228 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:50.953609 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:50.953612 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:50.953613 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:50.953671 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:50.953871 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:50.953872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:50.953872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:50.954017 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:50.954105 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:50.954272 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 16:41:50.954408 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":125,"thread_start_us":79,"threads_started":1}
I20260812 16:41:50.955106 11271 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.253.252" port: 38839 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:41:50.955482 11273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:50.955368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:38839 (local address 127.0.0.1:60452)
0812 16:41:50.955418 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:50.955463 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.252:38839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260812 16:41:50.955612 11188 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.253.252:38839: connect: Connection refused (error 111)
I20260812 16:41:50.956077 11273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:50.955954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:38839 (local address 127.0.0.1:60456)
0812 16:41:50.956007 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:50.956044 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.252:38839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 16:41:50.956187 11271 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.253.252:38839: Network error: Client connection negotiation failed: client connection to 127.0.253.252:38839: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.253.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.252.keytab.
Entry for principal kudu/127.0.253.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.252.keytab.
I20260812 16:41:50.965410 1015 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJXkJy5/build/release/bin/kudu
/tmp/dist-test-taskJXkJy5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.252:38839
--webserver_interface=127.0.253.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.252.keytab
--principal=kudu/127.0.253.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.253.212:40577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.253.254:34605,127.0.253.253:45221,127.0.253.252:38839
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:41:50.997287 11131 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.253.253" port: 45221 } attempt: 3
Aug 12 16:41:50 dist-test-slave-w4v5 krb5kdc[10994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552910, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for kudu/127.0.253.253@KRBTEST.COM
I20260812 16:41:51.004974 11134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:50.997700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.253:45221 (local address 127.0.0.1:40982)
0812 16:41:50.997778 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:50.997810 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:50.997868 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:50.998025 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:50.998026 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:50.998207 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:41:50.998274 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:50.998277 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:50.998950 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:50.998952 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:50.999613 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:50.999616 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:51.000039 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:51.000040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:51.000130 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:51.000610 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:41:51.000618 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:41:51.002056 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:41:51.003858 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:51.003859 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:51.003859 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:51.004047 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:51.004277 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:51.004278 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:51.004278 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:51.004343 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:51.004623 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:41:51.004624 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:41:51.004675 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 16:41:51.004764 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 16:41:51.004871 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260812 16:41:51.004974 11276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:50.997779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40982 (local address 127.0.253.253:45221)
0812 16:41:50.997851 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:50.997854 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:50.997866 (+ 12us) server_negotiation.cc:407] Connection header received
0812 16:41:50.997937 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:50.997938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:50.997957 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:50.998019 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:50.998364 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:50.998864 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:50.999704 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:50.999932 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:51.000039 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:51.002150 (+ 2111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:51.002152 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:51.002152 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:51.002160 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:51.003722 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:51.004135 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:51.004137 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:51.004137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:51.004199 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:51.004426 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:51.004427 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:51.004428 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:51.004552 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:51.004632 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:51.004790 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 16:41:51.004875 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 16:41:51.005402 11271 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.253.252" port: 38839 } attempt: 1
I20260812 16:41:51.005541 11131 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.253.252" port: 38839 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:41:51.005761 11273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:51.005651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:38839 (local address 127.0.0.1:60466)
0812 16:41:51.005696 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:51.005725 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.252:38839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260812 16:41:51.005903 11134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:51.005779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:38839 (local address 127.0.0.1:60470)
0812 16:41:51.005840 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:51.005876 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.252:38839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 16:41:51.006208 11273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:51.006115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:38839 (local address 127.0.0.1:60482)
0812 16:41:51.006152 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:51.006182 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.252:38839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260812 16:41:51.006291 11271 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.253.252:38839: Network error: Client connection negotiation failed: client connection to 127.0.253.252:38839: connect: Connection refused (error 111)
I20260812 16:41:51.006328 11134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:51.006180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:38839 (local address 127.0.0.1:60494)
0812 16:41:51.006234 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:51.006303 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.252:38839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 16:41:51.006412 11131 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.253.252:38839: Network error: Client connection negotiation failed: client connection to 127.0.253.252:38839: connect: Connection refused (error 111)
I20260812 16:41:51.037580 11131 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.253.252" port: 38839 } attempt: 1
I20260812 16:41:51.038100 11134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:51.037940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:38839 (local address 127.0.0.1:60502)
0812 16:41:51.038015 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:51.038061 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.252:38839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 16:41:51.038450 11134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:51.038364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:38839 (local address 127.0.0.1:60504)
0812 16:41:51.038395 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:51.038425 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.252:38839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260812 16:41:51.038684 11131 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.253.252:38839: Network error: Client connection negotiation failed: client connection to 127.0.253.252:38839: connect: Connection refused (error 111)
W20260812 16:41:51.051298 11281 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 16:41:51.051510 11281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:41:51.051527 11281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:41:51.051549 11281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:41:51.052618 11281 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:41:51.052654 11281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:41:51.052667 11281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:41:51.052680 11281 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:41:51.052692 11281 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:41:51.054106 11281 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.253.212:40577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.253.254:34605,127.0.253.253:45221,127.0.253.252:38839
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/krb5kdc/kudu_127.0.253.252.keytab
--principal=kudu/127.0.253.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.253.252:38839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.253.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:41:51.054306 11281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:41:51.054473 11281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:41:51.059379 11291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:41:51.059419 11295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:51.059473 11281 server_base.cc:1034] running on GCE node
W20260812 16:41:51.059484 11292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:41:51.059841 11281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:41:51.060103 11281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:41:51.061269 11281 hybrid_clock.cc:648] HybridClock initialized: now 1786552911061247 us; error 37 us; skew 500 ppm
Aug 12 16:41:51 dist-test-slave-w4v5 krb5kdc[10994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552911, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:41:51.064066 11281 init.cc:377] Logged in from keytab as kudu/127.0.253.252@KRBTEST.COM (short username kudu)
I20260812 16:41:51.064550 11281 webserver.cc:460] Webserver started at http://127.0.253.252:41465/ using document root <none> and password file <none>
I20260812 16:41:51.064792 11281 fs_manager.cc:359] Metadata directory not provided
I20260812 16:41:51.064842 11281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:41:51.064942 11281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:41:51.065974 11281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-2/data/instance:
uuid: "2c159b7bdf02443fa30366d239e7da78"
format_stamp: "Formatted at 2026-08-12 16:41:51 on dist-test-slave-w4v5"
server_key: "2e50a8270d86a6f10718bbb0a6d602f1"
server_key_iv: "377371dbdde1a694bafbb6777ac8e2a9"
server_key_version: "encryptionkey@0"
I20260812 16:41:51.066285 11281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-2/wal/instance:
uuid: "2c159b7bdf02443fa30366d239e7da78"
format_stamp: "Formatted at 2026-08-12 16:41:51 on dist-test-slave-w4v5"
server_key: "2e50a8270d86a6f10718bbb0a6d602f1"
server_key_iv: "377371dbdde1a694bafbb6777ac8e2a9"
server_key_version: "encryptionkey@0"
I20260812 16:41:51.067713 11281 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:41:51.068454 11311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:41:51.068651 11281 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 16:41:51.068732 11281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-2/data,/tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-2/wal
uuid: "2c159b7bdf02443fa30366d239e7da78"
format_stamp: "Formatted at 2026-08-12 16:41:51 on dist-test-slave-w4v5"
server_key: "2e50a8270d86a6f10718bbb0a6d602f1"
server_key_iv: "377371dbdde1a694bafbb6777ac8e2a9"
server_key_version: "encryptionkey@0"
I20260812 16:41:51.068780 11281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:41:51.074435 11271 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.253.252" port: 38839 } attempt: 2
I20260812 16:41:51.075024 11273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:51.074892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:38839 (local address 127.0.0.1:60506)
0812 16:41:51.074952 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:51.074992 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.252:38839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 16:41:51.075426 11273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:51.075342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:38839 (local address 127.0.0.1:60522)
0812 16:41:51.075384 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:51.075408 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.253.252:38839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260812 16:41:51.075551 11271 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.253.252:38839: Network error: Client connection negotiation failed: client connection to 127.0.253.252:38839: connect: Connection refused (error 111)
I20260812 16:41:51.084256 11281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:41:51.086104 11281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:41:51.086264 11281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:41:51.090258 11281 rpc_server.cc:307] RPC server started. Bound to: 127.0.253.252:38839
I20260812 16:41:51.090317 11395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.253.252:38839 every 8 connection(s)
I20260812 16:41:51.090695 11281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-2/data/info.pb
I20260812 16:41:51.090969 1015 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJXkJy5/build/release/bin/kudu as pid 11281
I20260812 16:41:51.091128 1015 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJXkJy5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552619305671-1015-0/minicluster-data/master-2/wal/instance
I20260812 16:41:51.091369 1015 external_mini_cluster.cc:1428] Setting key 047a820d27ac8cdb2d32919a8cfc28db
I20260812 16:41:51.092072 11396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:41:51.094216 11396 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.253.254" port: 34605 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 16:41:51 dist-test-slave-w4v5 krb5kdc[10994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.253.252@KRBTEST.COM
Aug 12 16:41:51 dist-test-slave-w4v5 krb5kdc[10994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552911, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.252@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:41:51.102875 11412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:51.094861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.254:34605 (local address 127.0.0.1:58962)
0812 16:41:51.095097 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:51.095140 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:51.095244 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:51.095381 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:51.095384 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:51.095726 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:41:51.095822 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:51.095824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:51.096660 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:51.096661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:51.097588 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:51.097591 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:51.097999 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:51.098001 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:51.098076 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:51.098502 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:41:51.098510 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:41:51.099886 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:41:51.101525 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:51.101532 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:51.101534 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:51.101820 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:51.102031 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:51.102032 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:51.102032 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:51.102092 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:51.102384 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:41:51.102392 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:41:51.102465 (+ 73us) client_negotiation.cc:769] Sending connection context
0812 16:41:51.102557 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 16:41:51.102668 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":85,"threads_started":1}
I20260812 16:41:51.102881 11410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:51.091915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60538 (local address 127.0.253.252:38839)
0812 16:41:51.092097 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:51.092105 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:51.092153 (+ 48us) server_negotiation.cc:407] Connection header received
0812 16:41:51.092619 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:51.092626 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:51.092672 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:51.092786 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:51.093586 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:51.094694 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:51.095872 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:51.096485 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:51.096594 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:51.099463 (+ 2869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:51.099471 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:51.099471 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:51.099479 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:51.101202 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:51.101573 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:51.101575 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:51.101575 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:51.101654 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:51.101885 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:51.101886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:51.101886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:51.102205 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:51.102360 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:51.102470 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 16:41:51.102611 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":100,"threads_started":1}
I20260812 16:41:51.103153 11135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:51.094929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58962 (local address 127.0.253.254:34605)
0812 16:41:51.095001 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:51.095003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:51.095242 (+ 239us) server_negotiation.cc:407] Connection header received
0812 16:41:51.095305 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:51.095306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:51.095316 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:51.095363 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:51.095898 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:51.096568 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:51.097681 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:51.097893 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:51.097947 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:51.099954 (+ 2007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:51.099955 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:51.099956 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:51.099964 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:51.101447 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:51.101899 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:51.101900 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:51.101900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:51.101955 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:51.102158 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:51.102159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:51.102159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:51.102307 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:51.102377 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:51.102958 (+ 581us) server_negotiation.cc:299] Negotiation successful
0812 16:41:51.103046 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 16:41:51.103817 11396 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.253.253" port: 45221 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 16:41:51 dist-test-slave-w4v5 krb5kdc[10994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552911, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.252@KRBTEST.COM for kudu/127.0.253.253@KRBTEST.COM
Aug 12 16:41:51 dist-test-slave-w4v5 krb5kdc[10994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.253.254@KRBTEST.COM
I20260812 16:41:51.111131 11276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:51.104200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40988 (local address 127.0.253.253:45221)
0812 16:41:51.104243 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:51.104245 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:51.104252 (+ 7us) server_negotiation.cc:407] Connection header received
0812 16:41:51.104275 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:51.104275 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:51.104284 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:51.104330 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:51.104635 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:51.105097 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:51.105773 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:51.105997 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:51.106074 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:51.108032 (+ 1958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:51.108034 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:51.108034 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:51.108042 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:51.109905 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:51.110258 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:51.110259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:51.110259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:51.110322 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:51.110537 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:51.110538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:51.110538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:51.110703 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:51.110786 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:51.110926 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 16:41:51.111004 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 16:41:51.111125 11412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:51.104066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.253:45221 (local address 127.0.0.1:40988)
0812 16:41:51.104098 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:51.104107 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:51.104133 (+ 26us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:51.104366 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:51.104367 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:51.104510 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:41:51.104567 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:51.104569 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:51.105184 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:51.105185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:51.105662 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:51.105664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:51.106071 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:51.106072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:51.106141 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:51.106634 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:41:51.106638 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:41:51.107940 (+ 1302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:41:51.109996 (+ 2056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:51.109997 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:51.109997 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:51.110176 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:51.110402 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:51.110403 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:51.110403 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:51.110466 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:51.110777 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:41:51.110778 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:41:51.110826 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 16:41:51.110900 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 16:41:51.111002 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260812 16:41:51.111507 11396 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.253.252" port: 38839 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:41:51.111912 11135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:51.103795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58964 (local address 127.0.253.254:34605)
0812 16:41:51.103838 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:51.103840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:51.103889 (+ 49us) server_negotiation.cc:407] Connection header received
0812 16:41:51.103955 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:51.103956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:51.103966 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:51.104020 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:51.104587 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:51.105015 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:51.105683 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:51.105891 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:51.105962 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:51.108518 (+ 2556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:51.108520 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:51.108520 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:51.108527 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:51.110180 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:51.110607 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:51.110608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:51.110609 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:51.110665 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:51.110904 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:51.110905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:51.110906 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:51.111020 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:51.111663 (+ 643us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:51.111715 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 16:41:51.111812 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 16:41:51.113859 11131 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.253.252" port: 38839 } attempt: 2
Aug 12 16:41:51 dist-test-slave-w4v5 krb5kdc[10994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552911, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.252@KRBTEST.COM for kudu/127.0.253.252@KRBTEST.COM
Aug 12 16:41:51 dist-test-slave-w4v5 krb5kdc[10994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.253.253@KRBTEST.COM
I20260812 16:41:51.118963 11412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:51.111801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:38839 (local address 127.0.0.1:60540)
0812 16:41:51.111854 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:51.111865 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:51.111901 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:51.112143 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:51.112144 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:51.112263 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:41:51.112317 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:51.112319 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:51.112999 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:51.113001 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:51.113582 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:51.113584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:51.114064 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:51.114064 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:51.114111 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:51.114541 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:41:51.114547 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:41:51.115949 (+ 1402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:41:51.117785 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:51.117786 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:51.117786 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:51.118000 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:51.118194 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:51.118195 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:51.118195 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:51.118271 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:51.118572 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:41:51.118573 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:41:51.118610 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 16:41:51.118679 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 16:41:51.118862 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":89}
Aug 12 16:41:51 dist-test-slave-w4v5 krb5kdc[10994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552910, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.254@KRBTEST.COM for kudu/127.0.253.252@KRBTEST.COM
I20260812 16:41:51.119742 11410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:51.111852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60540 (local address 127.0.253.252:38839)
0812 16:41:51.111889 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:51.111891 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:51.111899 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:41:51.111950 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:51.111951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:51.111963 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:51.112025 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:51.112389 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:51.112916 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:51.113660 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:51.113941 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:51.114020 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:51.116032 (+ 2012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:51.116034 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:51.116034 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:51.116041 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:51.117697 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:51.118073 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:51.118074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:51.118074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:51.118127 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:51.118342 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:51.118343 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:51.118343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:51.118481 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:51.118548 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:51.118704 (+ 156us) server_negotiation.cc:299] Negotiation successful
0812 16:41:51.118810 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 16:41:51.121006 11276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:51.112583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40996 (local address 127.0.253.253:45221)
0812 16:41:51.112624 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:51.112626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:51.112634 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:41:51.112654 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:51.112655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:51.112663 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:51.112711 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:51.113245 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:51.113713 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:51.114665 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:51.114984 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:51.115057 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:51.117823 (+ 2766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:51.117824 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:51.117825 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:51.117832 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:51.119612 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:51.119991 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:51.119992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:51.119993 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:51.120058 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:51.120269 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:51.120270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:51.120271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:51.120414 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:51.120594 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:51.120828 (+ 234us) server_negotiation.cc:299] Negotiation successful
0812 16:41:51.120905 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 16:41:51.121542 11396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c159b7bdf02443fa30366d239e7da78: Bootstrap starting.
I20260812 16:41:51.122349 11396 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c159b7bdf02443fa30366d239e7da78: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:51.122694 11396 log.cc:824] T 00000000000000000000000000000000 P 2c159b7bdf02443fa30366d239e7da78: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:51.122994 11134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:51.114114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:38839 (local address 127.0.0.1:60542)
0812 16:41:51.114170 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:51.114179 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:51.114214 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:51.114785 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:51.114786 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:51.114930 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:41:51.114964 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:51.114966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:51.116549 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:51.116551 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:51.117361 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:51.117364 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:51.117881 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:51.117883 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:51.117945 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:51.118504 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:41:51.118510 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:41:51.119853 (+ 1343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:41:51.121852 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:51.121853 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:51.121854 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:51.122077 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:51.122289 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:51.122290 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:51.122291 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:51.122361 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:51.122671 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:41:51.122672 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:41:51.122722 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 16:41:51.122798 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 16:41:51.122899 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 16:41:51.123005 11413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:51.114156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60542 (local address 127.0.253.252:38839)
0812 16:41:51.114663 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:51.114665 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:51.114673 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:41:51.114693 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:51.114693 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:51.114704 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:51.114777 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:51.115053 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:51.116470 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:51.117436 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:51.117785 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:51.117875 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:51.119942 (+ 2067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:51.119943 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:51.119943 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:51.119949 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:51.121759 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:51.122166 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:51.122167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:51.122167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:51.122219 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:51.122442 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:51.122443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:51.122443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:51.122588 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:51.122642 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:51.122839 (+ 197us) server_negotiation.cc:299] Negotiation successful
0812 16:41:51.122918 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":422,"threads_started":1}
I20260812 16:41:51.123788 11396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c159b7bdf02443fa30366d239e7da78: No bootstrap required, opened a new log
I20260812 16:41:51.124277 1015 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:41:51.124751 11131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24a1302bbbeb4ad59f83a6cc9e812209: Bootstrap starting.
I20260812 16:41:51.125492 11131 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24a1302bbbeb4ad59f83a6cc9e812209: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:51.125052 11097 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "24a1302bbbeb4ad59f83a6cc9e812209"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58964
I20260812 16:41:51.125803 11131 log.cc:824] T 00000000000000000000000000000000 P 24a1302bbbeb4ad59f83a6cc9e812209: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:51.125942 11396 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c159b7bdf02443fa30366d239e7da78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24a1302bbbeb4ad59f83a6cc9e812209" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 34605 } } peers { permanent_uuid: "4352a30003a84cdf8700225af97e098d" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 45221 } } peers { permanent_uuid: "2c159b7bdf02443fa30366d239e7da78" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 38839 } }
I20260812 16:41:51.126101 11396 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c159b7bdf02443fa30366d239e7da78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c159b7bdf02443fa30366d239e7da78, State: Initialized, Role: FOLLOWER
I20260812 16:41:51.126187 11396 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c159b7bdf02443fa30366d239e7da78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24a1302bbbeb4ad59f83a6cc9e812209" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 34605 } } peers { permanent_uuid: "4352a30003a84cdf8700225af97e098d" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 45221 } } peers { permanent_uuid: "2c159b7bdf02443fa30366d239e7da78" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 38839 } }
I20260812 16:41:51.126466 11419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c159b7bdf02443fa30366d239e7da78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24a1302bbbeb4ad59f83a6cc9e812209" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 34605 } } peers { permanent_uuid: "4352a30003a84cdf8700225af97e098d" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 45221 } } peers { permanent_uuid: "2c159b7bdf02443fa30366d239e7da78" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 38839 } } }
I20260812 16:41:51.126551 11396 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c159b7bdf02443fa30366d239e7da78 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:41:51.126574 11419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c159b7bdf02443fa30366d239e7da78 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:41:51.126822 11131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24a1302bbbeb4ad59f83a6cc9e812209: No bootstrap required, opened a new log
I20260812 16:41:51.128322 11131 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24a1302bbbeb4ad59f83a6cc9e812209 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24a1302bbbeb4ad59f83a6cc9e812209" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 34605 } } peers { permanent_uuid: "4352a30003a84cdf8700225af97e098d" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 45221 } } peers { permanent_uuid: "2c159b7bdf02443fa30366d239e7da78" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 38839 } }
I20260812 16:41:51.128463 11131 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24a1302bbbeb4ad59f83a6cc9e812209 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24a1302bbbeb4ad59f83a6cc9e812209, State: Initialized, Role: FOLLOWER
I20260812 16:41:51.128546 11131 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24a1302bbbeb4ad59f83a6cc9e812209 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24a1302bbbeb4ad59f83a6cc9e812209" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 34605 } } peers { permanent_uuid: "4352a30003a84cdf8700225af97e098d" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 45221 } } peers { permanent_uuid: "2c159b7bdf02443fa30366d239e7da78" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 38839 } }
W20260812 16:41:51.128713 11443 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2c159b7bdf02443fa30366d239e7da78: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 16:41:51.128762 11443 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:41:51.128886 11131 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24a1302bbbeb4ad59f83a6cc9e812209 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:41:51.128872 11446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24a1302bbbeb4ad59f83a6cc9e812209 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24a1302bbbeb4ad59f83a6cc9e812209" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 34605 } } peers { permanent_uuid: "4352a30003a84cdf8700225af97e098d" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 45221 } } peers { permanent_uuid: "2c159b7bdf02443fa30366d239e7da78" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 38839 } } }
I20260812 16:41:51.128952 11097 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24a1302bbbeb4ad59f83a6cc9e812209 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 16:41:51.129019 11097 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24a1302bbbeb4ad59f83a6cc9e812209 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:51.129920 11097 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24a1302bbbeb4ad59f83a6cc9e812209 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24a1302bbbeb4ad59f83a6cc9e812209" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 34605 } } peers { permanent_uuid: "4352a30003a84cdf8700225af97e098d" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 45221 } } peers { permanent_uuid: "2c159b7bdf02443fa30366d239e7da78" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 38839 } }
I20260812 16:41:51.130556 11361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "24a1302bbbeb4ad59f83a6cc9e812209" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2c159b7bdf02443fa30366d239e7da78"
I20260812 16:41:51.130683 11361 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c159b7bdf02443fa30366d239e7da78 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:51.128630 11281 master_runner.cc:186] Error getting master registration for 127.0.253.254:34605: OK, instance_id { permanent_uuid: "24a1302bbbeb4ad59f83a6cc9e812209" instance_seqno: 1786552910769385 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 16:41:51.130853 11281 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 16:41:51.131734 11361 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2c159b7bdf02443fa30366d239e7da78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24a1302bbbeb4ad59f83a6cc9e812209 in term 1.
I20260812 16:41:51.130215 11097 leader_election.cc:288] T 00000000000000000000000000000000 P 24a1302bbbeb4ad59f83a6cc9e812209 [CANDIDATE]: Term 1 election: Requested vote from peers 4352a30003a84cdf8700225af97e098d (127.0.253.253:45221), 2c159b7bdf02443fa30366d239e7da78 (127.0.253.252:38839)
I20260812 16:41:51.128949 11446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24a1302bbbeb4ad59f83a6cc9e812209 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 16:41:51.131762 11464 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 24a1302bbbeb4ad59f83a6cc9e812209: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260812 16:41:51.132102 11052 leader_election.cc:302] T 00000000000000000000000000000000 P 24a1302bbbeb4ad59f83a6cc9e812209 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24a1302bbbeb4ad59f83a6cc9e812209, 2c159b7bdf02443fa30366d239e7da78; no voters:
I20260812 16:41:51.132565 11446 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24a1302bbbeb4ad59f83a6cc9e812209 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:41:51.132714 11446 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24a1302bbbeb4ad59f83a6cc9e812209 [term 1 LEADER]: Becoming Leader. State: Replica: 24a1302bbbeb4ad59f83a6cc9e812209, State: Running, Role: LEADER
I20260812 16:41:51.132822 11446 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24a1302bbbeb4ad59f83a6cc9e812209 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24a1302bbbeb4ad59f83a6cc9e812209" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 34605 } } peers { permanent_uuid: "4352a30003a84cdf8700225af97e098d" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 45221 } } peers { permanent_uuid: "2c159b7bdf02443fa30366d239e7da78" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 38839 } }
I20260812 16:41:51.133283 11446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24a1302bbbeb4ad59f83a6cc9e812209 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24a1302bbbeb4ad59f83a6cc9e812209. Latest consensus state: current_term: 1 leader_uuid: "24a1302bbbeb4ad59f83a6cc9e812209" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24a1302bbbeb4ad59f83a6cc9e812209" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 34605 } } peers { permanent_uuid: "4352a30003a84cdf8700225af97e098d" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 45221 } } peers { permanent_uuid: "2c159b7bdf02443fa30366d239e7da78" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 38839 } } }
I20260812 16:41:51.133374 11446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24a1302bbbeb4ad59f83a6cc9e812209 [sys.catalog]: This master's current role is: LEADER
W20260812 16:41:51.133708 11464 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:41:51.134330 11022 master_runner.cc:186] Error getting master registration for 127.0.253.253:45221: OK, instance_id { permanent_uuid: "4352a30003a84cdf8700225af97e098d" instance_seqno: 1786552910931619 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 16:41:51.136890 11022 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 16:41:51.139277 11479 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:41:51.139516 11479 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:41:51.141692 11361 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2c159b7bdf02443fa30366d239e7da78 [term 1 FOLLOWER]: Refusing update from remote peer 24a1302bbbeb4ad59f83a6cc9e812209: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:41:51.142071 11446 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 24a1302bbbeb4ad59f83a6cc9e812209 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c159b7bdf02443fa30366d239e7da78" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 38839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:41:51.143882 11419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c159b7bdf02443fa30366d239e7da78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24a1302bbbeb4ad59f83a6cc9e812209. Latest consensus state: current_term: 1 leader_uuid: "24a1302bbbeb4ad59f83a6cc9e812209" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24a1302bbbeb4ad59f83a6cc9e812209" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 34605 } } peers { permanent_uuid: "4352a30003a84cdf8700225af97e098d" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 45221 } } peers { permanent_uuid: "2c159b7bdf02443fa30366d239e7da78" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 38839 } } }
I20260812 16:41:51.143980 11419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c159b7bdf02443fa30366d239e7da78 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:41:51.156421 11469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24a1302bbbeb4ad59f83a6cc9e812209 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "24a1302bbbeb4ad59f83a6cc9e812209" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24a1302bbbeb4ad59f83a6cc9e812209" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 34605 } } peers { permanent_uuid: "4352a30003a84cdf8700225af97e098d" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 45221 } } peers { permanent_uuid: "2c159b7bdf02443fa30366d239e7da78" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 38839 } } }
I20260812 16:41:51.156561 11469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24a1302bbbeb4ad59f83a6cc9e812209 [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:51.157253 11419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c159b7bdf02443fa30366d239e7da78 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "24a1302bbbeb4ad59f83a6cc9e812209" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24a1302bbbeb4ad59f83a6cc9e812209" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 34605 } } peers { permanent_uuid: "4352a30003a84cdf8700225af97e098d" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 45221 } } peers { permanent_uuid: "2c159b7bdf02443fa30366d239e7da78" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 38839 } } }
I20260812 16:41:51.157348 11419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c159b7bdf02443fa30366d239e7da78 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:41:51.157917 11413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:51.141123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60552 (local address 127.0.253.252:38839)
0812 16:41:51.141176 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:51.141179 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:51.147415 (+ 6236us) server_negotiation.cc:407] Connection header received
0812 16:41:51.147478 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:51.147479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:51.147492 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:51.147558 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:51.149013 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:51.149765 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:51.151527 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:51.151879 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:51.151948 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:51.154180 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:51.154182 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:51.154182 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:51.154199 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:51.156008 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:51.156596 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:51.156597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:51.156598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:51.156666 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:51.156962 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:51.156963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:51.156963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:51.157116 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:51.157213 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:51.157728 (+ 515us) server_negotiation.cc:299] Negotiation successful
0812 16:41:51.157825 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 16:41:51.158294 11485 mvcc.cc:205] Tried to move back new op lower bound from 7317720724032512000 to 7317720724000669696. Current Snapshot: MvccSnapshot[applied={T|T < 7317720724032512000}]
I20260812 16:41:51.158589 11480 mvcc.cc:205] Tried to move back new op lower bound from 7317720724032512000 to 7317720724000669696. Current Snapshot: MvccSnapshot[applied={T|T < 7317720724032512000}]
I20260812 16:41:51.160750 11479 catalog_manager.cc:1347] Generated new cluster ID: 5a6014473fb747a0bbe62fa4c2aa225c
I20260812 16:41:51.160805 11479 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:41:51.162354 11276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:51.140640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41010 (local address 127.0.253.253:45221)
0812 16:41:51.140693 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:51.140695 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:51.147241 (+ 6546us) server_negotiation.cc:407] Connection header received
0812 16:41:51.147339 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:51.147340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:51.147355 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:51.147419 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:51.148309 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:51.149173 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:51.150333 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:51.150717 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:51.150769 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:51.159263 (+ 8494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:51.159265 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:51.159266 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:51.159277 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:51.161056 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:51.161519 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:51.161520 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:51.161521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:51.161579 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:51.161782 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:51.161782 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:51.161783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:51.161910 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:51.161978 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:51.162176 (+ 198us) server_negotiation.cc:299] Negotiation successful
0812 16:41:51.162258 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 16:41:51.164839 11135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:51.141317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58970 (local address 127.0.253.254:34605)
0812 16:41:51.141397 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:51.141399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:51.149108 (+ 7709us) server_negotiation.cc:407] Connection header received
0812 16:41:51.149211 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:51.149213 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:51.149226 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:51.149262 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:51.151640 (+ 2378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:51.152423 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:51.154279 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:51.154596 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:51.154643 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:51.161399 (+ 6756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:51.161401 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:51.161401 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:51.161409 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:51.163199 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:51.163787 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:51.163799 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:51.163799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:51.163858 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:51.164138 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:51.164139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:51.164139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:51.164287 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:51.164379 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:51.164669 (+ 290us) server_negotiation.cc:299] Negotiation successful
0812 16:41:51.164750 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 16:41:51.166713 11271 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.253.252" port: 38839 } attempt: 3
Aug 12 16:41:51 dist-test-slave-w4v5 krb5kdc[10994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552910, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.253.253@KRBTEST.COM for kudu/127.0.253.252@KRBTEST.COM
I20260812 16:41:51.176784 11273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:51.166986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.253.252:38839 (local address 127.0.0.1:60564)
0812 16:41:51.167092 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0812 16:41:51.167102 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:41:51.167138 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:41:51.167420 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:41:51.167421 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:41:51.167582 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:41:51.167626 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:51.167628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:51.169146 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:51.169147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:51.169940 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:41:51.169943 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:41:51.170508 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:41:51.170510 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:41:51.170579 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:51.171125 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:41:51.171131 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:41:51.172705 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:41:51.174750 (+ 2045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:51.174751 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:51.174752 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:51.174966 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:51.175937 (+ 971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:41:51.175938 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:41:51.175938 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:41:51.176018 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:41:51.176473 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:41:51.176474 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:41:51.176529 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 16:41:51.176585 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 16:41:51.176680 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260812 16:41:51.177291 11413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:51.167229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60564 (local address 127.0.253.252:38839)
0812 16:41:51.167280 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:51.167281 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:51.167288 (+ 7us) server_negotiation.cc:407] Connection header received
0812 16:41:51.167319 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:51.167320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:51.167329 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:51.167374 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:51.167730 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:51.168463 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:51.170047 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:51.170370 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:51.170427 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:51.172808 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:41:51.172809 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:41:51.172809 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:41:51.172815 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:41:51.174652 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:51.175046 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:51.175047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:51.175048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:51.175158 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:41:51.176099 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:41:51.176100 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:41:51.176100 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:41:51.176350 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:41:51.176956 (+ 606us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:51.177028 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 16:41:51.177155 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 16:41:51.179023 11271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4352a30003a84cdf8700225af97e098d: Bootstrap starting.
I20260812 16:41:51.179809 11271 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4352a30003a84cdf8700225af97e098d: Neither blocks nor log segments found. Creating new log.
I20260812 16:41:51.180120 11271 log.cc:824] T 00000000000000000000000000000000 P 4352a30003a84cdf8700225af97e098d: Log is configured to *not* fsync() on all Append() calls
I20260812 16:41:51.191957 11271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4352a30003a84cdf8700225af97e098d: No bootstrap required, opened a new log
I20260812 16:41:51.194199 11271 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4352a30003a84cdf8700225af97e098d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24a1302bbbeb4ad59f83a6cc9e812209" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 34605 } } peers { permanent_uuid: "4352a30003a84cdf8700225af97e098d" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 45221 } } peers { permanent_uuid: "2c159b7bdf02443fa30366d239e7da78" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 38839 } }
I20260812 16:41:51.194386 11271 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4352a30003a84cdf8700225af97e098d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4352a30003a84cdf8700225af97e098d, State: Initialized, Role: FOLLOWER
I20260812 16:41:51.194483 11271 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4352a30003a84cdf8700225af97e098d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24a1302bbbeb4ad59f83a6cc9e812209" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 34605 } } peers { permanent_uuid: "4352a30003a84cdf8700225af97e098d" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 45221 } } peers { permanent_uuid: "2c159b7bdf02443fa30366d239e7da78" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 38839 } }
I20260812 16:41:51.194876 11271 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4352a30003a84cdf8700225af97e098d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:41:51.195005 11228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "24a1302bbbeb4ad59f83a6cc9e812209" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4352a30003a84cdf8700225af97e098d"
I20260812 16:41:51.195169 11228 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4352a30003a84cdf8700225af97e098d [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:41:51.196180 11228 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4352a30003a84cdf8700225af97e098d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24a1302bbbeb4ad59f83a6cc9e812209 in term 1.
I20260812 16:41:51.196337 11227 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4352a30003a84cdf8700225af97e098d [term 1 FOLLOWER]: Refusing update from remote peer 24a1302bbbeb4ad59f83a6cc9e812209: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:41:51.196825 11469 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 24a1302bbbeb4ad59f83a6cc9e812209 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4352a30003a84cdf8700225af97e098d" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 45221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 16:41:51.200441 11515 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4352a30003a84cdf8700225af97e098d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 16:41:51.200505 11515 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:41:51.200762 11500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4352a30003a84cdf8700225af97e098d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "24a1302bbbeb4ad59f83a6cc9e812209" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24a1302bbbeb4ad59f83a6cc9e812209" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 34605 } } peers { permanent_uuid: "4352a30003a84cdf8700225af97e098d" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 45221 } } peers { permanent_uuid: "2c159b7bdf02443fa30366d239e7da78" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 38839 } } }
I20260812 16:41:51.200847 11500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4352a30003a84cdf8700225af97e098d [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:41:51.200897 11500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4352a30003a84cdf8700225af97e098d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24a1302bbbeb4ad59f83a6cc9e812209. Latest consensus state: current_term: 1 leader_uuid: "24a1302bbbeb4ad59f83a6cc9e812209" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24a1302bbbeb4ad59f83a6cc9e812209" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 34605 } } peers { permanent_uuid: "4352a30003a84cdf8700225af97e098d" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 45221 } } peers { permanent_uuid: "2c159b7bdf02443fa30366d239e7da78" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 38839 } } }
I20260812 16:41:51.200948 11500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4352a30003a84cdf8700225af97e098d [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:41:51.200986 11500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4352a30003a84cdf8700225af97e098d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "24a1302bbbeb4ad59f83a6cc9e812209" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24a1302bbbeb4ad59f83a6cc9e812209" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 34605 } } peers { permanent_uuid: "4352a30003a84cdf8700225af97e098d" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 45221 } } peers { permanent_uuid: "2c159b7bdf02443fa30366d239e7da78" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 38839 } } }
I20260812 16:41:51.201045 11500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4352a30003a84cdf8700225af97e098d [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:41:51.201843 11469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24a1302bbbeb4ad59f83a6cc9e812209 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "24a1302bbbeb4ad59f83a6cc9e812209" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24a1302bbbeb4ad59f83a6cc9e812209" member_type: VOTER last_known_addr { host: "127.0.253.254" port: 34605 } } peers { permanent_uuid: "4352a30003a84cdf8700225af97e098d" member_type: VOTER last_known_addr { host: "127.0.253.253" port: 45221 } } peers { permanent_uuid: "2c159b7bdf02443fa30366d239e7da78" member_type: VOTER last_known_addr { host: "127.0.253.252" port: 38839 } } }
I20260812 16:41:51.201939 11469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24a1302bbbeb4ad59f83a6cc9e812209 [sys.catalog]: This master's current role is: LEADER
I20260812 16:41:51.202978 11479 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:41:51.203152 11496 mvcc.cc:205] Tried to move back new op lower bound from 7317720724032512000 to 7317720724000669696. Current Snapshot: MvccSnapshot[applied={T|T < 7317720724032512000}]
I20260812 16:41:51.203886 11479 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:41:51.212661 11479 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 24a1302bbbeb4ad59f83a6cc9e812209: Generated new TSK 0
I20260812 16:41:51.212910 11479 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 16:41:51.241422 11276 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:41:51.239555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41016 (local address 127.0.253.253:45221)
0812 16:41:51.239605 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:51.239607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:51.240639 (+ 1032us) server_negotiation.cc:407] Connection header received
0812 16:41:51.240755 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:51.240756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:51.240769 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:51.240801 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:51.241345 (+ 544us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41016: BlockingRecv error: recv got EOF from 127.0.0.1:41016 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260812 16:41:51.242317 11413 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:41:51.239976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60570 (local address 127.0.253.252:38839)
0812 16:41:51.240019 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:51.240021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:51.241746 (+ 1725us) server_negotiation.cc:407] Connection header received
0812 16:41:51.241849 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:51.241850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:51.241869 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:51.241899 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:51.242253 (+ 354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60570: BlockingRecv error: recv got EOF from 127.0.0.1:60570 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 16:41:51.243275 11135 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:41:51.240355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58986 (local address 127.0.253.254:34605)
0812 16:41:51.240396 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:51.240398 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:51.242589 (+ 2191us) server_negotiation.cc:407] Connection header received
0812 16:41:51.242777 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:51.242778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:51.242793 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:51.242823 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:51.243211 (+ 388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58986: BlockingRecv error: recv got EOF from 127.0.0.1:58986 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 16:41:51.245820 11276 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:41:51.244996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41018 (local address 127.0.253.253:45221)
0812 16:41:51.245033 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:51.245035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:51.245275 (+ 240us) server_negotiation.cc:407] Connection header received
0812 16:41:51.245391 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:51.245392 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:51.245412 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:51.245443 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:51.245775 (+ 332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41018: BlockingRecv error: recv got EOF from 127.0.0.1:41018 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 16:41:51.248855 11276 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:41:51.248079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41026 (local address 127.0.253.253:45221)
0812 16:41:51.248113 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:51.248115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:51.248344 (+ 229us) server_negotiation.cc:407] Connection header received
0812 16:41:51.248443 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:51.248444 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:51.248455 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:51.248485 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:51.248808 (+ 323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41026: BlockingRecv error: recv got EOF from 127.0.0.1:41026 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260812 16:41:51.252792 11276 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:41:51.251969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41030 (local address 127.0.253.253:45221)
0812 16:41:51.252005 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:51.252007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:51.252271 (+ 264us) server_negotiation.cc:407] Connection header received
0812 16:41:51.252380 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:51.252381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:51.252392 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:51.252423 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:51.252746 (+ 323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41030: BlockingRecv error: recv got EOF from 127.0.0.1:41030 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 16:41:51.258767 11276 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:41:51.257893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41036 (local address 127.0.253.253:45221)
0812 16:41:51.257933 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:51.257934 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:51.258210 (+ 276us) server_negotiation.cc:407] Connection header received
0812 16:41:51.258322 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:51.258323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:51.258334 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:51.258364 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:51.258703 (+ 339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41036: BlockingRecv error: recv got EOF from 127.0.0.1:41036 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 16:41:51.269129 11276 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:41:51.268150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41044 (local address 127.0.253.253:45221)
0812 16:41:51.268223 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:51.268225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:51.268521 (+ 296us) server_negotiation.cc:407] Connection header received
0812 16:41:51.268633 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:51.268634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:51.268646 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:51.268678 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:51.269072 (+ 394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41044: BlockingRecv error: recv got EOF from 127.0.0.1:41044 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260812 16:41:51.287758 11276 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:41:51.286604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41058 (local address 127.0.253.253:45221)
0812 16:41:51.286653 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:51.286655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:51.286986 (+ 331us) server_negotiation.cc:407] Connection header received
0812 16:41:51.287152 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:51.287154 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:51.287169 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:51.287202 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:51.287692 (+ 490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41058: BlockingRecv error: recv got EOF from 127.0.0.1:41058 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260812 16:41:51.321995 11276 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:41:51.321173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41062 (local address 127.0.253.253:45221)
0812 16:41:51.321222 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:51.321225 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:51.321387 (+ 162us) server_negotiation.cc:407] Connection header received
0812 16:41:51.321462 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:51.321463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:51.321477 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:51.321551 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:51.321922 (+ 371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41062: BlockingRecv error: recv got EOF from 127.0.0.1:41062 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260812 16:41:51.388087 11276 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:41:51.387272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41076 (local address 127.0.253.253:45221)
0812 16:41:51.387345 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:51.387347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:51.387466 (+ 119us) server_negotiation.cc:407] Connection header received
0812 16:41:51.387546 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:51.387547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:51.387566 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:51.387644 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:51.388013 (+ 369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41076: BlockingRecv error: recv got EOF from 127.0.0.1:41076 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260812 16:41:51.518280 11276 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:41:51.517318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41080 (local address 127.0.253.253:45221)
0812 16:41:51.517387 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:51.517388 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:51.517675 (+ 287us) server_negotiation.cc:407] Connection header received
0812 16:41:51.517759 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:51.517760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:51.517774 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:51.517849 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:51.518207 (+ 358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41080: BlockingRecv error: recv got EOF from 127.0.0.1:41080 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260812 16:41:51.776281 11276 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:41:51.775566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41090 (local address 127.0.253.253:45221)
0812 16:41:51.775634 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:51.775636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:51.775650 (+ 14us) server_negotiation.cc:407] Connection header received
0812 16:41:51.775744 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:51.775745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:51.775755 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:51.775807 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:41:51.776209 (+ 402us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41090: BlockingRecv error: recv got EOF from 127.0.0.1:41090 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 16:41:52.129339 11443 catalog_manager.cc:1259] Loaded cluster ID: 5a6014473fb747a0bbe62fa4c2aa225c
I20260812 16:41:52.129423 11443 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2c159b7bdf02443fa30366d239e7da78: loading cluster ID for follower catalog manager: success
I20260812 16:41:52.130390 11443 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2c159b7bdf02443fa30366d239e7da78: acquiring CA information for follower catalog manager: success
I20260812 16:41:52.130738 11443 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2c159b7bdf02443fa30366d239e7da78: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 16:41:52.200954 11515 catalog_manager.cc:1259] Loaded cluster ID: 5a6014473fb747a0bbe62fa4c2aa225c
I20260812 16:41:52.201062 11515 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4352a30003a84cdf8700225af97e098d: loading cluster ID for follower catalog manager: success
I20260812 16:41:52.201905 11515 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4352a30003a84cdf8700225af97e098d: acquiring CA information for follower catalog manager: success
I20260812 16:41:52.202268 11515 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4352a30003a84cdf8700225af97e098d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 16:41:52.291859 11637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:52.289373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41106 (local address 127.0.253.253:45221)
0812 16:41:52.289482 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:52.289484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:52.289497 (+ 13us) server_negotiation.cc:407] Connection header received
0812 16:41:52.289593 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:52.289594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:52.289648 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:52.289748 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:41:52.289880 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:52.290371 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:52.291122 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:52.291312 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:52.291364 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:52.291503 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:41:52.291613 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:41:52.291673 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:52.291720 (+ 47us) server_negotiation.cc:299] Negotiation successful
0812 16:41:52.291737 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":50,"threads_started":1}
I20260812 16:41:52.296391 11649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:52.293774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60586 (local address 127.0.253.252:38839)
0812 16:41:52.293899 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:52.293901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:52.293911 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:41:52.293988 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:52.293989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:52.294031 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:52.294119 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:41:52.294244 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:52.294741 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:52.295550 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:52.295783 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:52.295860 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:52.296042 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:41:52.296143 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:41:52.296233 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:52.296273 (+ 40us) server_negotiation.cc:299] Negotiation successful
0812 16:41:52.296295 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":54,"threads_started":1}
I20260812 16:41:52.300824 11649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:52.298158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60596 (local address 127.0.253.252:38839)
0812 16:41:52.298227 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:52.298229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:52.298398 (+ 169us) server_negotiation.cc:407] Connection header received
0812 16:41:52.298431 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:52.298432 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:52.298445 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:52.298503 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:41:52.298605 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:52.299376 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:52.300145 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:52.300354 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:52.300429 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:52.300578 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:41:52.300608 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:41:52.300655 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:52.300718 (+ 63us) server_negotiation.cc:299] Negotiation successful
0812 16:41:52.300736 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 16:41:52.301564 11637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:52.298079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41110 (local address 127.0.253.253:45221)
0812 16:41:52.298131 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:52.298133 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:52.298432 (+ 299us) server_negotiation.cc:407] Connection header received
0812 16:41:52.298623 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:52.298624 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:52.298636 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:52.298665 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:41:52.298874 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:52.299614 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:52.300729 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:52.301018 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:52.301057 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:52.301284 (+ 227us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:41:52.301324 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:41:52.301364 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:52.301436 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 16:41:52.301452 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 16:41:52.301777 11662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:41:52.298078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59002 (local address 127.0.253.254:34605)
0812 16:41:52.298313 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 16:41:52.298315 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:41:52.298444 (+ 129us) server_negotiation.cc:407] Connection header received
0812 16:41:52.298496 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:41:52.298498 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:41:52.298551 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:41:52.298624 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:41:52.298792 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:52.299557 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:52.300725 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:41:52.301051 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:41:52.301114 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:41:52.301308 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:41:52.301422 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:41:52.301487 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 16:41:52.301645 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 16:41:52.301672 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":57,"threads_started":1}
I20260812 16:41:52.303552 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 11022
I20260812 16:41:52.320253 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 11144
I20260812 16:41:52.332741 1015 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJXkJy5/build/release/bin/kudu with pid 11281
2026-08-12T16:41:52Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3173 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6373 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (293039 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST